{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "bOChJSNXtC9g"
   },
   "source": [
    "# Embeddings"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "OLIxEDq6VhvZ"
   },
   "source": [
    "So far, we've represented text in a bagged one-hot encoded form which is a n-dimensional array where each index corresponds to a token. The value at that index corresponds to the number of times the word appears in the sentence. This method forces us to completely lose the structural information in our inputs. \n",
    "\n",
    "```python\n",
    "[0. 0. 1. 0. 0. 0. 0. 0. 0. 0. 0. 1. 0. 0. 0. 0. 0. 1. 0. 0. 0. 0. 0. 0.\n",
    " 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 1. 0. 0. 0. 0. 0. 0. 0.\n",
    " 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
    " 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]```\n",
    " \n",
    " We've also represented our input in a one-hot encoded form where each token is represented by an n-dimensional array. T\n",
    " \n",
    " ```python\n",
    "[[0. 0. 0. ... 0. 0. 0.]\n",
    " [0. 0. 1. ... 0. 0. 0.]\n",
    " [0. 0. 0. ... 0. 0. 0.]\n",
    " ...\n",
    " [0. 0. 0. ... 0. 0. 0.]\n",
    " [0. 0. 0. ... 0. 0. 0.]\n",
    " [0. 0. 0. ... 0. 0. 0.]]\n",
    "```\n",
    "\n",
    "his allows us to preserve the structural information but there are two major disadvantages here. If we have a large vocabulary, the representation length for each token will be massive leading to large computes. And though we preserve the structure within the text, the actual representation for each token does not preserve any relationship with respect to other tokens.\n",
    "\n",
    "In this notebook, we're going to learn about embeddings and how they address all the shortcomings of the representation methods we've seen so far.\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "VoMq0eFRvugb"
   },
   "source": [
    "# Overview"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "qWro5T5qTJJL"
   },
   "source": [
    "* **Objective:**  Represent tokens in text that capture the intrinsic semantic relationships.\n",
    "* **Advantages:** \n",
    "    * Low-dimensionality while capturing relationships.\n",
    "    * Interpretable token representations\n",
    "* **Disadvantages:** None\n",
    "* **Miscellaneous:** There are lot's of pretrained embeddings to choose from but you can also train your own from scratch."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "2thIKTLYwzJd"
   },
   "source": [
    "# Learning embeddings"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "MlxioJLqx2Ls"
   },
   "source": [
    "The main idea of embeddings is to have fixed length representations for the tokens in a text regardless of the number of tokens in the vocabulary. So instead of each token representation having the shape [1XV] where V is vocab size, each token now has the shape [1 X D] where D is the embedding size (usually 50, 100, 200, 300). The numbers in the representation will no longer be 0s and 1s but rather floats that represent that token in a D-dimensional latent space. If the embeddings really did capture the relationship between tokens, then we should be able to inspect this latent space and confirm known relationships (we'll do this soon).\n",
    "\n",
    "But how do we learn the embeddings the first place? The intuition behind embeddings is that the definition of a token depends on the token itself but on it's context. There are several different ways of doing this:\n",
    "\n",
    "1. Given the word in the context, predict the target word (CBOW - continuous bag of words).\n",
    "2. Given the target word, predict the context word (skip-gram).\n",
    "3. Given a sequence of words, predict the next word (LM - language modeling).\n",
    "\n",
    "All of these approaches involve create data to train our model on. Every word in a sentence becomes the target word and the context words are determines by a window. In the image below (skip-gram), the window size is 2. We repeat this for every sentence in our corpus and this results in our training data for the unsupervised task. This in an unsupervised learning technique since we don't have official labels for contexts. The idea is that similar target words will appear with similar contexts and we can learn this relationship by repeatedly training our mode with (context, target) pairs.\n",
    "\n",
    "![](https://raw.githubusercontent.com/GokuMohandas/practicalAI/master/images/skipgram.png?raw=true)\n",
    "\n",
    "We can learn embeddings using any of these approaches above and some work better than others. You can inspect the learned embeddings but the best way to choose an approach is to empirically validate the performance on a supervised task. We can learn embeddings by creating our models in PyTorch but instead, we're going to use a library that specializes in embeddings and topic modeling called [Gensim](https://radimrehurek.com/gensim/). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 343
    },
    "colab_type": "code",
    "id": "uGDGEVvz41LL",
    "outputId": "42f2b9c0-07d2-42ec-dc7f-2fe91bb4f875"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: gensim in /usr/local/lib/python3.6/dist-packages (3.6.0)\n",
      "Requirement already satisfied: numpy>=1.11.3 in /usr/local/lib/python3.6/dist-packages (from gensim) (1.14.6)\n",
      "Requirement already satisfied: smart-open>=1.2.1 in /usr/local/lib/python3.6/dist-packages (from gensim) (1.7.1)\n",
      "Requirement already satisfied: scipy>=0.18.1 in /usr/local/lib/python3.6/dist-packages (from gensim) (1.1.0)\n",
      "Requirement already satisfied: six>=1.5.0 in /usr/local/lib/python3.6/dist-packages (from gensim) (1.11.0)\n",
      "Requirement already satisfied: boto>=2.32 in /usr/local/lib/python3.6/dist-packages (from smart-open>=1.2.1->gensim) (2.49.0)\n",
      "Requirement already satisfied: bz2file in /usr/local/lib/python3.6/dist-packages (from smart-open>=1.2.1->gensim) (0.98)\n",
      "Requirement already satisfied: requests in /usr/local/lib/python3.6/dist-packages (from smart-open>=1.2.1->gensim) (2.18.4)\n",
      "Requirement already satisfied: boto3 in /usr/local/lib/python3.6/dist-packages (from smart-open>=1.2.1->gensim) (1.9.67)\n",
      "Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests->smart-open>=1.2.1->gensim) (3.0.4)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.6/dist-packages (from requests->smart-open>=1.2.1->gensim) (2018.11.29)\n",
      "Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python3.6/dist-packages (from requests->smart-open>=1.2.1->gensim) (2.6)\n",
      "Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python3.6/dist-packages (from requests->smart-open>=1.2.1->gensim) (1.22)\n",
      "Requirement already satisfied: botocore<1.13.0,>=1.12.67 in /usr/local/lib/python3.6/dist-packages (from boto3->smart-open>=1.2.1->gensim) (1.12.67)\n",
      "Requirement already satisfied: s3transfer<0.2.0,>=0.1.10 in /usr/local/lib/python3.6/dist-packages (from boto3->smart-open>=1.2.1->gensim) (0.1.13)\n",
      "Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in /usr/local/lib/python3.6/dist-packages (from boto3->smart-open>=1.2.1->gensim) (0.9.3)\n",
      "Requirement already satisfied: docutils>=0.10 in /usr/local/lib/python3.6/dist-packages (from botocore<1.13.0,>=1.12.67->boto3->smart-open>=1.2.1->gensim) (0.14)\n",
      "Requirement already satisfied: python-dateutil<3.0.0,>=2.1; python_version >= \"2.7\" in /usr/local/lib/python3.6/dist-packages (from botocore<1.13.0,>=1.12.67->boto3->smart-open>=1.2.1->gensim) (2.5.3)\n"
     ]
    }
   ],
   "source": [
    "!pip install gensim "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 51
    },
    "colab_type": "code",
    "id": "9wr9S6965DD7",
    "outputId": "e527ab0c-559c-4cf2-bdca-cfc554b723f2"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[nltk_data] Downloading package punkt to /root/nltk_data...\n",
      "[nltk_data]   Unzipping tokenizers/punkt.zip.\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "from argparse import Namespace\n",
    "import copy\n",
    "import gensim\n",
    "from gensim.models import Word2Vec\n",
    "import json\n",
    "import nltk; nltk.download('punkt')\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import re\n",
    "import urllib\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "5-sx-n9655TJ"
   },
   "outputs": [],
   "source": [
    "args = Namespace(\n",
    "    seed=1234,\n",
    "    data_file=\"harrypotter.txt\",\n",
    "    embedding_dim=100,\n",
    "    window=5,\n",
    "    min_count=3,\n",
    "    skip_gram=1, # 0 = CBOW\n",
    "    negative_sampling=20,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "VaJSRgNEg_V_"
   },
   "outputs": [],
   "source": [
    "# Upload data from GitHub to notebook's local drive\n",
    "url = \"https://raw.githubusercontent.com/GokuMohandas/practicalAI/master/data/harrypotter.txt\"\n",
    "response = urllib.request.urlopen(url)\n",
    "html = response.read()\n",
    "with open(args.data_file, 'wb') as fp:\n",
    "    fp.write(html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 51
    },
    "colab_type": "code",
    "id": "irvgngOG5yqk",
    "outputId": "8be95f0e-38fd-456f-df57-245b713a8927"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "15640\n",
      "Snape nodded, but did not elaborate.\n"
     ]
    }
   ],
   "source": [
    "# Split text into sentences\n",
    "tokenizer = nltk.data.load('tokenizers/punkt/english.pickle')\n",
    "with open(args.data_file, encoding='cp1252') as fp:\n",
    "    book = fp.read()\n",
    "sentences = tokenizer.tokenize(book)\n",
    "print (len(sentences))\n",
    "print (sentences[11])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "VTREFDg47Vrx"
   },
   "outputs": [],
   "source": [
    "# Preprocessing\n",
    "def preprocess_text(text):\n",
    "    text = ' '.join(word.lower() for word in text.split(\" \"))\n",
    "    text = re.sub(r\"([.,!?])\", r\" \\1 \", text)\n",
    "    text = re.sub(r\"[^a-zA-Z.,!?]+\", r\" \", text)\n",
    "    text = text.strip()\n",
    "    return text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "id": "IX7pkCbg7WPK",
    "outputId": "d1789fb1-3952-427d-bb4f-684c71397fff"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "snape nodded , but did not elaborate .\n"
     ]
    }
   ],
   "source": [
    "# Clean sentences\n",
    "sentences = [preprocess_text(sentence) for sentence in sentences]\n",
    "print (sentences[11])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "id": "dsmEEVzX5ytO",
    "outputId": "14eaadd8-318f-463b-ae38-fda714014083"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['snape', 'nodded', ',', 'but', 'did', 'not', 'elaborate', '.']\n"
     ]
    }
   ],
   "source": [
    "# Process sentences for gensim\n",
    "sentences = [sentence.split(\" \") for sentence in sentences]\n",
    "print (sentences[11])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Fa-DbjPW-KC3"
   },
   "source": [
    "When we have large vocabularies to learn embeddings for, things can get complex very quickly. Recall that the backpropagation with softmax updates both the correct and incorrect class weights. This becomes a massive computation for every backwas pass we do so a workaround is to use [negative sampling](http://mccormickml.com/2017/01/11/word2vec-tutorial-part-2-negative-sampling/) which only updates the correct class and a few arbitrary incorrect classes (negative_sampling=20). We're able to do this because of the large amount of training data where we'll see the same word as the target class multiple times.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "id": "wzkUkl-b5ywC",
    "outputId": "6c165235-fc86-42f3-b47d-fad93f903f8f"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Word2Vec(vocab=4837, size=100, alpha=0.025)\n"
     ]
    }
   ],
   "source": [
    "# Super fast because of optimized C code under the hood\n",
    "model = Word2Vec(sentences=sentences, size=args.embedding_dim, \n",
    "                 window=args.window, min_count=args.min_count, \n",
    "                 sg=args.skip_gram, negative=args.negative_sampling)\n",
    "print (model)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 374
    },
    "colab_type": "code",
    "id": "gcMb9HUd7vS_",
    "outputId": "6233793a-61a8-4f21-b83e-327ddb8abea4"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-0.22578861, -0.08212412,  0.23737124, -0.15245605,  0.36689585,\n",
       "        0.25724858, -0.01634605, -0.1440076 , -0.19065167,  0.03596726,\n",
       "        0.19535664,  0.13759346, -0.15498197, -0.02432883,  0.49592218,\n",
       "        0.14719264, -0.07807084,  0.04079456,  0.35241577, -0.11118395,\n",
       "       -0.00079209,  0.10813674, -0.44349238,  0.09216332, -0.35282937,\n",
       "        0.0060242 , -0.26499394, -0.20232336,  0.17964938,  0.03854164,\n",
       "        0.03296188,  0.17841353,  0.51603127, -0.1633792 ,  0.55372894,\n",
       "        0.5718708 ,  0.57195616, -0.2596567 , -0.09128902,  0.17179367,\n",
       "       -0.16872773, -0.27405336,  0.16995263, -0.1405293 ,  0.39514822,\n",
       "        0.11885013, -0.17025688,  0.2911798 ,  0.38346636, -0.26334456,\n",
       "       -0.14385992,  0.05425156,  0.37470722,  0.01140373, -0.1725289 ,\n",
       "        0.4729884 , -0.23966838,  0.09094231, -0.21025181,  0.08470887,\n",
       "        0.4726624 , -0.02922293, -0.5122037 ,  0.49850512,  0.18162192,\n",
       "       -0.16878986, -0.28319255, -0.6642979 ,  0.22183757,  0.09444458,\n",
       "       -0.0328227 , -0.12481013,  0.19649623,  0.04153791,  0.4201806 ,\n",
       "       -0.13673969, -0.44183746, -0.18778716,  0.01571143,  0.17307448,\n",
       "       -0.07169608,  0.10320378,  0.09087852,  0.01220326,  0.23870039,\n",
       "       -0.27214712,  0.21593754,  0.00388517, -0.00279438, -0.11385468,\n",
       "        0.24877366, -0.46467814,  0.05003679, -0.15469068,  0.3235739 ,\n",
       "       -0.04771227, -0.00554706, -0.12149673, -0.10426747,  0.24046072],\n",
       "      dtype=float32)"
      ]
     },
     "execution_count": 10,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Vector for each word\n",
    "model.wv.get_vector(\"potter\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 102
    },
    "colab_type": "code",
    "id": "BozlP5WG70Ak",
    "outputId": "d066a258-f789-43f5-e285-11a2655dbe3d"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('pain', 0.9334665536880493),\n",
       " ('forehead', 0.9138509035110474),\n",
       " ('burning', 0.912197470664978),\n",
       " ('burned', 0.9033104181289673),\n",
       " ('mouth', 0.8993061780929565)]"
      ]
     },
     "execution_count": 11,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Get nearest neighbors (excluding itself)\n",
    "model.wv.most_similar(positive=\"scar\", topn=5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "q-wgfMnH68vQ"
   },
   "outputs": [],
   "source": [
    "# Save the weights \n",
    "model.wv.save_word2vec_format('model.txt', binary=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "K1OC2smgw1P5"
   },
   "source": [
    "# Pretrained embeddings"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "DHUU_-TJz_7K"
   },
   "source": [
    "We can learn embeddings from scratch using one of the approaches above but we can also leverage pretrained embeddings that have been trained on millions of documents. Popular ones include Word2Vec (skip-gram) or GloVe (global word-word co-occurrence). We can validate that these embeddings captured meaningful semantic relationships by confirming them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "vJNngmZjglVg"
   },
   "outputs": [],
   "source": [
    "from gensim.scripts.glove2word2vec import glove2word2vec\n",
    "from gensim.models import KeyedVectors\n",
    "from io import BytesIO\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.decomposition import PCA\n",
    "from zipfile import ZipFile\n",
    "from urllib.request import urlopen"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 85
    },
    "colab_type": "code",
    "id": "5WZ2bT27gncI",
    "outputId": "528ddc8e-d8e4-4ce9-d376-c31adb2e00e1"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['glove.6B.50d.txt',\n",
       " 'glove.6B.100d.txt',\n",
       " 'glove.6B.200d.txt',\n",
       " 'glove.6B.300d.txt']"
      ]
     },
     "execution_count": 14,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Unzip the file (may take ~3 minutes)\n",
    "resp = urlopen('http://nlp.stanford.edu/data/glove.6B.zip')\n",
    "zipfile = ZipFile(BytesIO(resp.read()))\n",
    "zipfile.namelist()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "id": "NNacKuRNi8_x",
    "outputId": "9416cec2-387a-4025-8c65-a1075119f6ce"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'/content/glove.6B.100d.txt'"
      ]
     },
     "execution_count": 15,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Write embeddings\n",
    "embeddings_file = 'glove.6B.{0}d.txt'.format(args.embedding_dim)\n",
    "zipfile.extract(embeddings_file)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "id": "SA5Y1BGbbpPo",
    "outputId": "9c4191ce-84ad-4cf8-cb99-7baa9c379a96"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(400000, 100)"
      ]
     },
     "execution_count": 16,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Save GloVe embeddings to local directory in word2vec format\n",
    "word2vec_output_file = '{0}.word2vec'.format(embeddings_file)\n",
    "glove2word2vec(embeddings_file, word2vec_output_file)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "bQp1pIJLiZw3"
   },
   "outputs": [],
   "source": [
    "# Load embeddings (may take a minute)\n",
    "glove = KeyedVectors.load_word2vec_format(word2vec_output_file, binary=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 122
    },
    "colab_type": "code",
    "id": "HDpHnC0sbpVE",
    "outputId": "f65be2e7-1b52-4859-f514-c72efe7bc0ab"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('queen', 0.7698541283607483),\n",
       " ('monarch', 0.6843380928039551),\n",
       " ('throne', 0.6755735874176025),\n",
       " ('daughter', 0.6594556570053101),\n",
       " ('princess', 0.6520534753799438)]"
      ]
     },
     "execution_count": 18,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# (king - man) + woman = ?\n",
    "glove.most_similar(positive=['woman', 'king'], negative=['man'], topn=5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 122
    },
    "colab_type": "code",
    "id": "_1TatbmvbpYU",
    "outputId": "6f6955bc-5f35-49f2-81a2-5b2b3e91ff99"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('gohan', 0.7246542572975159),\n",
       " ('bulma', 0.6497020125389099),\n",
       " ('raistlin', 0.6443604230880737),\n",
       " ('skaar', 0.6316742897033691),\n",
       " ('guybrush', 0.6231324672698975)]"
      ]
     },
     "execution_count": 19,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Get nearest neighbors (exlcusing itself)\n",
    "glove.wv.most_similar(positive=\"goku\", topn=5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "oMljB5MEbpdb"
   },
   "outputs": [],
   "source": [
    "# Reduce dimensionality for plotting\n",
    "X = glove[glove.wv.vocab]\n",
    "pca = PCA(n_components=2)\n",
    "pca_results = pca.fit_transform(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Aok5pRqXqcCf"
   },
   "outputs": [],
   "source": [
    "def plot_embeddings(words, embeddings, pca_results):\n",
    "    for word in words:\n",
    "        index = embeddings.index2word.index(word)\n",
    "        plt.scatter(pca_results[index, 0], pca_results[index, 1])\n",
    "        plt.annotate(word, xy=(pca_results[index, 0], pca_results[index, 1]))\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 347
    },
    "colab_type": "code",
    "id": "NOdLFgTOrrBd",
    "outputId": "60dd126b-565c-42ae-bf2c-ff0d203b2aab"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeQAAAFKCAYAAADMuCxnAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAHpJJREFUeJzt3Xt0VPW99/HPXHIhJIRJMhFMwgJB\nQLk1FG0jVDQGS5XHVoskFMW7h1Vp8YItBA9RMdwq6AJ81EZrVZDQ0ohSPaZVgnK4VFRAAo9CoiKX\nGCbkAkMIkGSeP6xz5BBmoklmfpl5v/7K7L1nft/5rKz5ZO8ZBovH4/EIAAAElTXYAwAAAAoZAAAj\nUMgAABiAQgYAwAAUMgAABqCQAQAwgD3QC7pcxwK9pJfDEaOamvqgrd8ZkJFv5OMfGflGPv6FYkZO\nZ5zfY8LqDNlutwV7BOORkW/k4x8Z+UY+/oVrRmFVyAAAmIpCBgDAABQyAAAGoJABADAAhQwAgAEo\nZAAADEAhw6ePPvpA2dm/OGPb7t2luv/+qUGaCABCE4WM7+ziiwdr8eJlwR4DAEJKwL+pC51XY2Oj\n7rvvHl122U+0Zs1qrVq1Rs8//6zq6mrlcrlUVrZX3bvHa968xUpKStKnn36ivLyZkqSrr/6Z3n13\nnaZNm67hw0cE+ZkAgHk4Q0arPfnkH5SW1ksDBgw8Y3tJyTuaNu0B/eUva+RwJOiNN16TJC1cmK/s\n7EkqLHxVsbGx2r//y2CMDQCdAoWMVnn11dU6cGC/7r//92ftGzYsXT169JTFYtGFFw5QZeVXOnmy\nQZ9++v+UlfVTSdINN0yQx+MJ9NgA0GlQyDink6ebVOs+qSNHjuiZZ5YqMTFRdvvZ73LExsZ6f7Za\nrWpubtaxY8dksVgUF/f1F6rb7XY5HAkBmx0AOhveQ8ZZmpqbtWpdmbbtcenAZ7vV6LHqxrvmad1f\nF+rdd0u8JetLTExXeTweNTQ0KDo6Wo2NjaqtrQnA9ADQOXGGjLOsWlemtz84oCNHT8ojyWKL1r/K\nGnTJ1bdr8eL5qq2t9fsYMTEx6t27j9at+6ck6bXXiiRZOnZwAOjEKGScoeFUo7btcbW4r/Jkkq64\n8motWjSvVY91//2/10sv/Uk33TRBDQ0n5HQ6ZbFQygDQEosnwJ+0cbmOBXK5MzidcUFdvzNotFj1\nH/PeVku/FFaLNPfuHyvZEdPqx/N4PN4SHjcuS0888X914YX922nawON3yD8y8o18/AvFjJxO/2/1\ncYaMMzi6RSmhW1TL++KiFR/b8r6WPPTQ77VixYuSpA8/3CqPx6NevXq1y5wAEGpaVch79uxRVlaW\nli9ffs5jFi1apJtvvrndBkNwREfald7f2eK+9P5Jioqwtfqx7rxzit57b71ycm7Qk0/+QQ899Kii\noqLba1QACCl+P2VdX1+vOXPmKCMj45zHlJWVaevWrYqIiGjX4RAc2Zn9JEnb9lSp5liDHHHRSu+f\n5N3eWr1799Ef//jnDpgQAEKP30KOjIxUQUGBCgoKznnM/Pnzdd9992nZMr7fOBTYrFb9Kqu/fjm6\nr+rcJxUfG/WdzowBAN+d30K22+0tfhnEN4qKinTppZcqJSWlVQs6HDGy24P34t6aN9bD3bczSg3i\nHKbid8g/MvKNfPwLx4za9MUgtbW1Kioq0gsvvKDKyspW3aempr4tS7ZJKH5yr72RkW/k4x8Z+UY+\n/oViRh3+KestW7aourpakyZN0tSpU7Vr1y7NnTu3LQ8JAEBYatMZ8tixYzV27FhJ0oEDBzRz5kzl\n5ua2y2AAAIQTv4VcWlqqBQsW6ODBg7Lb7SouLlZmZqZSU1M1ZsyYQMwIAEDI45u6cAYy8o18/CMj\n38jHv1DMiG/qAgCgk6CQAQAwAIUMAIABKGQAAAxAIQMAYAAKGQAAA1DIAAAYgEIGAMAAFDIAAAag\nkAEAMACFDACAAShkAAAMQCEDAGAAChkAAANQyAAAGIBCBgDAABQyAAAGoJABADAAhQwAgAEoZAAA\nDEAhAwBgAAoZAAADUMgAABiAQgYAwAAUMgAABqCQAQAwAIUMAIABKGQAAAxAIQMAYAAKGQAAA1DI\nAAAYgEIGAMAAFDIAAAagkAEAMACFDACAAShkAAAMQCEDAGAAChkAAANQyAAAGIBCBgDAABQyAAAG\noJABADAAhQwAgAEoZAAADEAhAwBgAAoZAAADUMgAABiAQgYAwAAUMgAABqCQAQAwAIUMAIABWlXI\ne/bsUVZWlpYvX37Wvi1btmjChAnKycnRzJkz1dzc3O5DAgAQ6vwWcn19vebMmaOMjIwW98+ePVtL\nlixRYWGhjh8/rg0bNrT7kAAAhDq/hRwZGamCggIlJye3uL+oqEg9evSQJCUkJKimpqZ9JwQAIAzY\n/R5gt8tuP/dhsbGxkqTDhw9r48aNmjZtms/HczhiZLfbvuOY7cfpjAva2p0FGflGPv6RkW/k4184\nZuS3kFvjyJEjmjJlivLy8uRwOHweW1NT3x5Lfi9OZ5xcrmNBW78zICPfyMc/MvKNfPwLxYxa8wdG\nmz9l7Xa7ddddd+nee+/VqFGj2vpwAACEpTYX8vz583XLLbfo8ssvb495AAAIS34vWZeWlmrBggU6\nePCg7Ha7iouLlZmZqdTUVI0aNUpr1qzRvn37tHr1aknSuHHjlJ2d3eGDAwAQSvwW8uDBg/Xyyy+f\nc39paWm7DgQAQDjim7oAADAAhQwAgAEoZAAADEAhAwBgAAoZAAADUMgAABiAQgYAwAAUMgAABqCQ\nAQAwAIUMAIABKGQAAAxAIQMAYAAKGQAAA1DIAAAYgEIGAMAAFDIAAAagkAEAMACFDACAAShkAAAM\nQCEDAGAAChkAAANQyAAAGIBCBgDAABQyAAAGoJABADAAhQwAgAEoZAAADEAhAwBgAHuwBwAAwCQV\nFYc0ZcptmjDhV/r731+TxyM99NAjevHF57R37x5deumPlZubp7Vr16iwcLmampqUmJik//zPR9Wj\nR0+9+eZabdr03+ratat27Nguu92mZcuW6sILL/S5LmfIAAD8L7W1tUpISNTKlUXq16+f8vJmatas\nR/Tiiyv19tvF2rWrVE88sVBPPPGUCgtfVUpKqv785+e899+yZaOuv/5GFRYWKT19hF588UW/a1LI\nAAD8W/PJkzp95IiampqUmZklSbrggn666KKL1b17d8XHd1diYpIaG0+ruPhdJSefJ0kaNixdhw4d\n9D5O794XaODAiyRJAwYMUEVFhd+1uWQNAAh7nqYmuf5aKPe2j/TVV1/JKqnu1b/JeWOOrFarunSJ\n8R5rtVp1+vRpPffcM9q48T01NTWpvr5eaWm9vMd07Rr7reNtampq8jsDhQwACHuuvxaq9u1/fn3D\n45Gk/7ndgurqam3c+J6WLStQ9+7d9frrr+of//ivNs3AJWsAQFhrPnlS7m0ftbjPvW2bPC2c3dbW\nVqtHj57q3r276upqtW7dP3XixIk2zUEhAwDCWmNdnRqrq1veV1Ot5oaGs7YnJiaprq5O2dm/0MMP\nz9Jdd/1ahw9XaunSJ773HBaP59/n5gHich0L5HJncDrjgrp+Z0BGvpGPf2TkG/n4F+iMmk+e1Bez\nc9V45MhZ++yJSer9aL6sUVFtWsPpjPN7DGfIAICwZo2KUmz68Bb3xaant7mMW4sPdQEAwp7zxhxJ\nX79n3FhTLbsjQbHp6d7tgUAhAwDCnsVmU3LOJCVdP16NdXWyx8cH7Mz4GxQyAAD/Zo2KUmRycnDW\nDsqqAADgDBQyAAAGoJABADAAhQwAgAEoZAAADEAhAwBgAAoZAAADUMgAABiAQgYAwAAUMgAABmhV\nIe/Zs0dZWVlavnz5Wfs2bdqk8ePHKzs7W0899VS7DwgAQDjwW8j19fWaM2eOMjIyWtz/2GOPaenS\npVq5cqU2btyosrKydh8SAIBQ57eQIyMjVVBQoOQWvmx7//79io+PV8+ePWW1WjV69Ght3ry5QwYF\nACCU+S1ku92u6OjoFve5XC4lJCR4byckJMjlcrXfdAAAhImA//eLDkeM7HZboJf1cjrjgrZ2Z0FG\nvpGPf2TkG/n4F44ZtamQk5OTVVVV5b1dWVnZ4qXtb6upqW/Lkm3idMbJ5ToWtPU7AzLyjXz8IyPf\nyMe/UMyoNX9gtOmfPaWmpsrtduvAgQNqbGxUSUmJRo4c2ZaHBAAgLPk9Qy4tLdWCBQt08OBB2e12\nFRcXKzMzU6mpqRozZowefvhhPfDAA5Kka665Rn369OnwoQEACDUWj8fjCeSCwbwMEYqXQdobGflG\nPv6RkW/k418oZtThl6wBAED7oJABADAAhQwAgAEoZAAADEAhAwBgAAoZAAADUMgAABiAQgYAwAAU\nMgAABqCQAQAwAIUMAIABKGQAAAxAIQMAYAAKGQAAA1DIAAAYgEIGAMAAFDIAAAagkAEAMACFDACA\nAShkAAAMQCEDAGAAChkAAANQyAAAGIBCBgDAABQyAAAGoJABADAAhQwAgAEoZAAADEAhAwBgAAoZ\nAAADUMgAABiAQgYAwAAUMgAABqCQAQAwAIUMAIABKGQAAAxAIQMAYAAKGQAAA1DIAAAYgEIGAMAA\nFDIAAAagkAEAMACFDACAAShkAAAMQCEDAGAAChkAAANQyAAAGIBCBgDAABQyAAAGoJABADAAhQwA\ngAHsrTlo7ty52rFjhywWi3JzczV06FDvvhUrVuj111+X1WrV4MGDNWvWrA4bFgCAUOX3DPn999/X\nvn37tGrVKuXn5ys/P9+7z+126/nnn9eKFSu0cuVKlZeXa/v27R06MAAAochvIW/evFlZWVmSpL59\n+6qurk5ut1uSFBERoYiICNXX16uxsVEnTpxQfHx8x04MAEAI8lvIVVVVcjgc3tsJCQlyuVySpKio\nKN1zzz3KysrSlVdeqWHDhqlPnz4dNy0AACGqVe8hf5vH4/H+7Ha79eyzz+qtt95SbGysbrnlFn3y\nyScaOHDgOe/vcMTIbrd9v2nbgdMZF7S1Owsy8o18/CMj38jHv3DMyG8hJycnq6qqynv78OHDcjqd\nkqTy8nKlpaUpISFBkjRixAiVlpb6LOSamvq2zvy9OZ1xcrmOBW39zoCMfCMf/8jIN/LxLxQzas0f\nGH4vWY8cOVLFxcWSpF27dik5OVmxsbGSpJSUFJWXl6uhoUGSVFpaqt69e7dhZAAAwpPfM+Thw4dr\n0KBBysnJkcViUV5enoqKihQXF6cxY8bojjvu0OTJk2Wz2ZSenq4RI0YEYm4AAEKKxfPtN4UDIJiX\nIULxMkh7IyPfyMc/MvKNfPwLxYza5ZI1AADoeBQyAAAGoJABADAAhQwAgAEoZAAADEAhAwBgAAoZ\nAAADUMgAABiAQgYAwAAUMgAABqCQAQAwAIUMAIABKGQAAAxAIQMAYAAKGQAAA1DIAAAYgEIGAMAA\nFDIAAAagkAEAMACFDACAAShkAAAMQCEDAGAAe7AHABD6brjhWi1Z8oxSU9P0zjv/0Jw5s/XWW+sV\nHR2twsLl2rdvn6xWiz766ANZrVb9+Mcj9etf/1Y2m03jx/8f5eRM0ptvrpXL5dL06TP0wQdb9a9/\nbVL37g49/vgSdevWTaWlH2vx4oVqaDghq9WqadOm65JLfqSKikOaMuU23XTTbVq79lUdPXpUv/nN\nfbrqqquDHQtwBs6QAXS49PQfqrT0Y0nS9u3bNGDARdq9u1SStGPHdjmdTh0+XKmXX/6L/vSn5fr4\n4216++1i7/0/+6xcf/rTCt166x2aM2e2rrzyKq1atUYeT7Pee2+dJGnhwnz96lc365VX/qZJk27R\n44/P896/trZWVqtFL720Sr/97QMqKHg6gM8eaB0KGUCHGz58hEpLd0qSdu3aqXHjfq6dO3dIknbv\n3qkNG9bruuuul91uV1RUtMaM+Znef3+L9/4/+ckVkqQLLuinqKgoDR8+QhaLRX369FVVVZUk6YUX\nXlFm5hhJ0rBh6Tp06KD3/k1NTbrmmuskSQMGDFRl5Vcd/pyB74pL1gA6zOnTTap3n9LQoelavbpQ\nR48eVUREhIYPH6Ennlioffu+UHJyDx0/7lZcXDfv/eLi4lRTU+O9HRPTVZJktdrUpUuMd7vValVT\nU5Mk6R//+C+tXr1K9fXH1dzcLI/H4z3OZrOpS5cu3vs0Nzd36PMGvg8KGUC7a25u1qZ15fp8T5Xc\nR08qtluUqo8c1ZYtGzV48BClpKSqouKQduzYphEjLtXOnTtUV1fnvf/Ro3VKSEho9Xou12EtXJiv\nP/7xz7rwwgHav/9LTZx4Q0c8NaDDcMkaQLvbtK5cOz84KPfRk5Ik99GTiu+apheef1FDhgyTJPXq\n1VtvvPG6fvjDS3TZZaP0xhuvqampSSdOnFBx8ZvKyBjV6vVqa2sUHd1FvXr1VmNjo15//VVJUn19\nffs/OaCDUMgA2tXpU436fE/VWdvPS+yn/QfLNHDgYEnSkCFDtXfvpxoyZJh++ctsJSefp5tvnqA7\n77xZl132E2VmZrV6zX79+isjY6QmTrxBU6bcrpEjf6JBg4Zo6tS72+15AR3N4vn2Gy0B4HIdC+Ry\nZ3A644K6fmdARr6Rj382i1XL5q1rcZ/FIk28+0eKd3QJ8FTm4HfIv1DMyOmM83sMZ8gA2lVctyjF\ndotqcV9sXJRiYiMDPBHQOVDIANpVRKRdffontbivd/8kRUTYAjwR0DnwKWsA7e6yzL6SpC/2VMl9\n7KRi46LUu3+SdzuAs1HIANqd1WrVqKwL9aPRF6jefUoxsZGcGQN+UMgAOkxEhC2sP8AFfBe8hwwA\ngAEoZAAADEAhAwBgAAoZAAADUMgAABiAQgYAwAAUMgAABqCQAQAwAIUMAIABKGQAAAxAIQMAYAAK\nGQAAA1DIAAAYgEIGAMAAFDIAAAagkAEAMIC9NQfNnTtXO3bskMViUW5uroYOHerdV1FRofvvv1+n\nT5/WxRdfrEcffbTDhgUAIFT5PUN+//33tW/fPq1atUr5+fnKz88/Y//8+fN1++23a/Xq1bLZbDp0\n6FCHDQsAQKjyW8ibN29WVlaWJKlv376qq6uT2+2WJDU3N+vDDz9UZmamJCkvL0/nn39+B44LAEBo\n8lvIVVVVcjgc3tsJCQlyuVySpOrqanXt2lXz5s3TxIkTtWjRoo6bFACAENaq95C/zePxnPFzZWWl\nJk+erJSUFN19991av369rrjiinPe3+GIkd1u+17DtgenMy5oa3cWZOQb+fhHRr6Rj3/hmJHfQk5O\nTlZVVZX39uHDh+V0OiVJDodD559/vnr16iVJysjI0N69e30Wck1NfRtH/v6czji5XMeCtn5nQEa+\nkY9/ZOQb+fgXihm15g8Mv5esR44cqeLiYknSrl27lJycrNjYWEmS3W5XWlqavvjiC+/+Pn36tGFk\nAADCk98z5OHDh2vQoEHKycmRxWJRXl6eioqKFBcXpzFjxig3N1czZsyQx+NR//79vR/wAgAArWfx\nfPtN4QAI5mWIULwM0t7IyDfy8Y+MfCMf/0Ixo3a5ZA0AADoehQwAgAEoZAAADEAhAwBgAAoZAAAD\nUMgAABiAQgYAwAAUMgAABqCQAQAwAIUMAIABKGQAAAzwnf8/ZFO98EKB1q5do/j4eGVmXq3XXvub\n0tN/qJSUVN16652SpBkzZigx8Tzdeuud+vzzz7Ro0XxVVVUpMjJCubl5GjjwYknSa68VadWqFTp1\n6pQGDRqi3NzZioqKVn7+w+rRo6d27tyh/fu/VFpaL82fv1jR0dHBfOoAgBAQEmfIn31WplWrXlFB\nwYsqKHhJu3fv9Hl8c3OzZs6crrFjr1FhYZGmT5+pGTMeUGNjo3bs2KbnnntGS5Y8o9Wr1yo2NlYF\nBc9471tS8rYefXSeVq1ao9raWr33XklHPz0AQBgIiUL++OPtSk8frsTEJNntdmVljfV5/L59X6i2\ntlrXXvtzSdLQoT9Q9+4OlZZ+rI0b39NVV41RUpJTkvSLX/xS7777P6WbkTFK3brFy263q2/fvqqs\n/KrjnhgAIGx0+kvWp5pOqeLIV4rp2tW7zeFw+LyP231MDQ0NmjRpvHfb8ePHVVdXp2PH3NqwoUTv\nv79FktTc7FFj42nvcV2/tY7ValNTU1N7PRUAQBjrtIXc1NykorI39LFrl/a6StWw/5j+uud13dDv\nWtXW1kqSrFarmpubvfepq6tTYuJ5SkpyqmvXrnrllb+d9bhlZXs0duw4TZ16b8CeCwAAnfaSdVHZ\nG1p/4L9VfbJGXdK6qeZzl97+dJ1Wf7pWb731d0lSYmKSysr2SpIOHjygjz76SJLUo0dPOZ3nqaTk\nbUlSbW2t8vJydeLECY0aNVrvvluimpoaSdKGDeu1fPmfA/8EAQBhpVOeIZ9qOqWPXbu8t2N6xinx\nh+drz9NbtS92p2782QR99lm5rrvueuXmTldOzvXq33+gfvrTn0qSLBaLHnlkrv7wh7kqKHhaVqtV\n2dmT1KVLFw0YMFCTJ9+m3/zmP+TxNMvhSNCDD+YG66kCAMKExePxeAK5oMt1rO2PUX9Ej2xZKI/O\nHt0ii26MvUZLFz6u1avXnrHP6Yxrl/VDGRn5Rj7+kZFv5ONfKGbkdMb5PaZTXrKOj4qTI6p7i/sS\norsrNiImwBMBANA2nbKQI22RGuoc1OK+IUmDZLdGBHgiAADaplO+hyxJN/S7VpK0s2qXqhtqlRDd\nXUOSBumGftfKZrWddbkaAACTddpCtllturH/dfp537GqO3lM8VFxirRFBnssAAC+l05byN+ItEXK\nGZMY7DEAAGiTTvkeMgAAoYZCBgDAABQyAAAGoJABADAAhQwAgAEoZAAADEAhAwBgAAoZAAADBPx/\newIAAGfjDBkAAANQyAAAGIBCBgDAABQyAAAGoJABADAAhQwAgAFCrpBPnDihadOm6aabbtKNN96o\nkpKSM/ZXVFRo4sSJGj9+vGbPnh2kKYPLX0YrVqxQdna2Jk6cqPz8/CBNGXwNDQ3KyspSUVHRGds3\nbdqk8ePHKzs7W0899VSQpjPDuTLasmWLJkyYoJycHM2cOVPNzc1BmjC4zpXPNxYtWqSbb745wFOZ\n5VwZheNrdcgVcklJiQYPHqzly5frySef1Pz588/YP3/+fN1+++1avXq1bDabDh06FKRJg8dXRm63\nW88//7xWrFihlStXqry8XNu3bw/itMHz9NNPKz4+/qztjz32mJYuXaqVK1dq48aNKisrC8J0ZjhX\nRrNnz9aSJUtUWFio48ePa8OGDUGYLvjOlY8klZWVaevWrQGeyDznyigcX6vtwR6gvV1zzTXenysq\nKnTeeed5bzc3N+vDDz/U4sWLJUl5eXkBn88EvjKKiIhQRESE6uvrFRMToxMnTpzzBSWUlZeXq6ys\nTFdcccUZ2/fv36/4+Hj17NlTkjR69Ght3rxZ/fr1C8KUwXWujCSpqKhIsbGxkqSEhATV1NQEeLrg\n85WP9HXh3HfffVq2bFlgBzPIuTIK19fqkDtD/kZOTo6mT5+u3Nxc77bq6mp17dpV8+bN08SJE7Vo\n0aIgThh8LWUUFRWle+65R1lZWbryyis1bNgw9enTJ4hTBseCBQs0Y8aMs7a7XC4lJCR4byckJMjl\ncgVyNGOcKyNJ3jI+fPiwNm7cqNGjRwdyNCP4yqeoqEiXXnqpUlJSAjyVWc6VUbi+VodsIRcWFurp\np5/Wgw8+qG++HdTj8aiyslKTJ0/W8uXLtXv3bq1fvz64gwZRSxm53W49++yzeuutt/TOO+9ox44d\n+uSTT4I8aWCtWbNGP/jBD5SWlhbsUYzVmoyOHDmiKVOmKC8vTw6HI4DTBZ+vfGpra1VUVKTbbrst\nCJOZw1dG4fpaHXKXrEtLS5WYmKiePXvqoosuUlNTk6qrq5WYmCiHw6Hzzz9fvXr1kiRlZGRo7969\n57ykFKp8ZVReXq60tDTvWeCIESNUWlqqgQMHBnnqwFm/fr3279+v9evX66uvvlJkZKR69Oihyy67\nTMnJyaqqqvIeW1lZqeTk5CBOGxy+MpK+/sPurrvu0r333qtRo0YFedrA85XPli1bVF1drUmTJunU\nqVP68ssvNXfu3DOuVIUDXxmF62t1yBXyBx98oIMHD2rWrFmqqqpSfX29969zu92utLQ0ffHFF+rd\nu7d27dqla6+9NsgTB56vjFJSUlReXq6GhgZFR0ertLQ07C43Pvnkk96fly5dqpSUFG/RpKamyu12\n68CBA+rRo4dKSkr0+OOPB2vUoPGVkfT1+6O33HKLLr/88mCMF3S+8hk7dqzGjh0rSTpw4IBmzpwZ\ndmUs+c4oXF+rQ+5/e2poaNCsWbNUUVGhhoYGTZ06VbW1tYqLi9OYMWO0b98+zZgxQx6PR/3799fD\nDz8sqzVkr9y3yF9GhYWFKioqks1mU3p6un73u98Fe+Sg+eaFQpI3n61bt3pL+Oqrr9Ydd9wRzBGD\n7n9nNGrUKF1yySVKT0/3HjNu3DhlZ2cHa8Sgaul36BvfFPLLL78crPGM0FJG4fhaHXKFDABAZxTa\nf24AANBJUMgAABiAQgYAwAAUMgAABqCQAQAwAIUMAIABKGQAAAxAIQMAYID/D1fKexaGj5blAAAA\nAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f70f2d1fa20>"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_embeddings(words=[\"king\", \"queen\", \"man\", \"woman\"], embeddings=glove, \n",
    "                pca_results=pca_results)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 102
    },
    "colab_type": "code",
    "id": "6667pZttboaj",
    "outputId": "1408c5c8-0195-4fea-ff11-ca4d8cc68806"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('nurse', 0.7735227346420288),\n",
       " ('physician', 0.7189429998397827),\n",
       " ('doctors', 0.6824328303337097),\n",
       " ('patient', 0.6750682592391968),\n",
       " ('dentist', 0.6726033687591553)]"
      ]
     },
     "execution_count": 23,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Bias in embeddings\n",
    "glove.most_similar(positive=['woman', 'doctor'], negative=['man'], topn=5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "LGOkD_DGIq7t"
   },
   "source": [
    "# Using Embeddings"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "tu3Bl-_yw9LA"
   },
   "source": [
    "There are several different ways to use embeddings. \n",
    "\n",
    "1. Use your own trained embeddings (trained on an unsupervised dataset).\n",
    "2. Use pretrained embeddings (GloVe, word2vec, etc.)\n",
    "3. Randomly initialized embeddings.\n",
    "\n",
    "Once you have chosen embeddings, you can choose to freeze them or continue to train them using the supervised data (this could lead to overfitting). In this example, we're going to use GloVe embeddings and freeze them during training. Our task will be to predict an article's category given its title."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "8OlZB0vc1Hbb"
   },
   "source": [
    "## Set up"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "id": "11CPv1Z-IrD6",
    "outputId": "7791dc95-e1c6-441d-8d7d-1381576c1554"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: torch in /usr/local/lib/python3.6/dist-packages (1.0.0)\n"
     ]
    }
   ],
   "source": [
    "# Load PyTorch library\n",
    "!pip3 install torch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "T3daO-e9wpXe"
   },
   "outputs": [],
   "source": [
    "import os\n",
    "from argparse import Namespace\n",
    "import collections\n",
    "import json\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import re\n",
    "import torch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Ye64bXPrwpaQ"
   },
   "outputs": [],
   "source": [
    "# Set Numpy and PyTorch seeds\n",
    "def set_seeds(seed, cuda):\n",
    "    np.random.seed(seed)\n",
    "    torch.manual_seed(seed)\n",
    "    if cuda:\n",
    "        torch.cuda.manual_seed_all(seed)\n",
    "        \n",
    "# Creating directories\n",
    "def create_dirs(dirpath):\n",
    "    if not os.path.exists(dirpath):\n",
    "        os.makedirs(dirpath)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "id": "m5_jiImZ1NAr",
    "outputId": "3b174e83-9e51-412e-bfab-02d3606be15e"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Using CUDA: True\n"
     ]
    }
   ],
   "source": [
    "# Arguments\n",
    "args = Namespace(\n",
    "    seed=1234,\n",
    "    cuda=True,\n",
    "    shuffle=True,\n",
    "    data_file=\"news.csv\",\n",
    "    vectorizer_file=\"vectorizer.json\",\n",
    "    model_state_file=\"model.pth\",\n",
    "    save_dir=\"news\",\n",
    "    train_size=0.7,\n",
    "    val_size=0.15,\n",
    "    test_size=0.15,\n",
    "    cutoff=25, # token must appear at least <cutoff> times to be in SequenceVocabulary\n",
    "    num_epochs=5,\n",
    "    early_stopping_criteria=5,\n",
    "    learning_rate=1e-3,\n",
    "    batch_size=64,\n",
    "    num_filters=100,\n",
    "    embedding_dim=100,\n",
    "    hidden_dim=100,\n",
    "    dropout_p=0.1,\n",
    ")\n",
    "\n",
    "# Set seeds\n",
    "set_seeds(seed=args.seed, cuda=args.cuda)\n",
    "\n",
    "# Create save dir\n",
    "create_dirs(args.save_dir)\n",
    "\n",
    "# Expand filepaths\n",
    "args.vectorizer_file = os.path.join(args.save_dir, args.vectorizer_file)\n",
    "args.model_state_file = os.path.join(args.save_dir, args.model_state_file)\n",
    "\n",
    "# Check CUDA\n",
    "if not torch.cuda.is_available():\n",
    "    args.cuda = False\n",
    "args.device = torch.device(\"cuda\" if args.cuda else \"cpu\")\n",
    "print(\"Using CUDA: {}\".format(args.cuda))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "GkNclM3I1XMg"
   },
   "source": [
    "## Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "vk6ucvgp1NDD"
   },
   "outputs": [],
   "source": [
    "import re\n",
    "import urllib"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "uVQNGEYa1NFu"
   },
   "outputs": [],
   "source": [
    "# Upload data from GitHub to notebook's local drive\n",
    "url = \"https://raw.githubusercontent.com/GokuMohandas/practicalAI/master/data/news.csv\"\n",
    "response = urllib.request.urlopen(url)\n",
    "html = response.read()\n",
    "with open(args.data_file, 'wb') as fp:\n",
    "    fp.write(html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 204
    },
    "colab_type": "code",
    "id": "aTU6ENCu1NIc",
    "outputId": "0e1a8e12-fd6e-4073-f4cb-47491aef5401"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>category</th>\n",
       "      <th>title</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Business</td>\n",
       "      <td>Wall St. Bears Claw Back Into the Black (Reuters)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Business</td>\n",
       "      <td>Carlyle Looks Toward Commercial Aerospace (Reu...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Business</td>\n",
       "      <td>Oil and Economy Cloud Stocks' Outlook (Reuters)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Business</td>\n",
       "      <td>Iraq Halts Oil Exports from Main Southern Pipe...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Business</td>\n",
       "      <td>Oil prices soar to all-time record, posing new...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   category                                              title\n",
       "0  Business  Wall St. Bears Claw Back Into the Black (Reuters)\n",
       "1  Business  Carlyle Looks Toward Commercial Aerospace (Reu...\n",
       "2  Business    Oil and Economy Cloud Stocks' Outlook (Reuters)\n",
       "3  Business  Iraq Halts Oil Exports from Main Southern Pipe...\n",
       "4  Business  Oil prices soar to all-time record, posing new..."
      ]
     },
     "execution_count": 30,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Raw data\n",
    "df = pd.read_csv(args.data_file, header=0)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 85
    },
    "colab_type": "code",
    "id": "xIouG0f71NK4",
    "outputId": "7e1cc683-157f-49f3-c853-ce0cd9410756"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Business: 30000\n",
      "Sci/Tech: 30000\n",
      "Sports: 30000\n",
      "World: 30000\n"
     ]
    }
   ],
   "source": [
    "# Split by category\n",
    "by_category = collections.defaultdict(list)\n",
    "for _, row in df.iterrows():\n",
    "    by_category[row.category].append(row.to_dict())\n",
    "for category in by_category:\n",
    "    print (\"{0}: {1}\".format(category, len(by_category[category])))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "YZyFpi791NNP"
   },
   "outputs": [],
   "source": [
    "# Create split data\n",
    "final_list = []\n",
    "for _, item_list in sorted(by_category.items()):\n",
    "    if args.shuffle:\n",
    "        np.random.shuffle(item_list)\n",
    "    n = len(item_list)\n",
    "    n_train = int(args.train_size*n)\n",
    "    n_val = int(args.val_size*n)\n",
    "    n_test = int(args.test_size*n)\n",
    "\n",
    "  # Give data point a split attribute\n",
    "    for item in item_list[:n_train]:\n",
    "        item['split'] = 'train'\n",
    "    for item in item_list[n_train:n_train+n_val]:\n",
    "        item['split'] = 'val'\n",
    "    for item in item_list[n_train+n_val:]:\n",
    "        item['split'] = 'test'  \n",
    "\n",
    "    # Add to final list\n",
    "    final_list.extend(item_list)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 85
    },
    "colab_type": "code",
    "id": "NLrQTVoO1NP4",
    "outputId": "8dd24897-f9c8-465f-dded-706faa84cbd2"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "train    84000\n",
       "test     18000\n",
       "val      18000\n",
       "Name: split, dtype: int64"
      ]
     },
     "execution_count": 33,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# df with split datasets\n",
    "split_df = pd.DataFrame(final_list)\n",
    "split_df[\"split\"].value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 204
    },
    "colab_type": "code",
    "id": "v-HSMGwh1NUu",
    "outputId": "1dc58e32-ae79-4ac1-ae6d-af19d0c22754"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>category</th>\n",
       "      <th>split</th>\n",
       "      <th>title</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Business</td>\n",
       "      <td>train</td>\n",
       "      <td>general electric posts higher rd quarter profit</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Business</td>\n",
       "      <td>train</td>\n",
       "      <td>lilly to eliminate up to us jobs</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Business</td>\n",
       "      <td>train</td>\n",
       "      <td>s amp p lowers america west outlook to negative</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Business</td>\n",
       "      <td>train</td>\n",
       "      <td>does rand walk the talk on labor policy ?</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Business</td>\n",
       "      <td>train</td>\n",
       "      <td>housekeeper advocates for changes</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   category  split                                            title\n",
       "0  Business  train  general electric posts higher rd quarter profit\n",
       "1  Business  train                 lilly to eliminate up to us jobs\n",
       "2  Business  train  s amp p lowers america west outlook to negative\n",
       "3  Business  train       does rand walk the talk on labor policy ? \n",
       "4  Business  train                housekeeper advocates for changes"
      ]
     },
     "execution_count": 34,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Preprocessing\n",
    "def preprocess_text(text):\n",
    "    text = ' '.join(word.lower() for word in text.split(\" \"))\n",
    "    text = re.sub(r\"([.,!?])\", r\" \\1 \", text)\n",
    "    text = re.sub(r\"[^a-zA-Z.,!?]+\", r\" \", text)\n",
    "    return text\n",
    "    \n",
    "split_df.title = split_df.title.apply(preprocess_text)\n",
    "split_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "cPDWGdlE5vye"
   },
   "source": [
    "## Vocabulary"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "EMR5Y3D95v6P"
   },
   "outputs": [],
   "source": [
    "class Vocabulary(object):\n",
    "    def __init__(self, token_to_idx=None):\n",
    "\n",
    "        # Token to index\n",
    "        if token_to_idx is None:\n",
    "            token_to_idx = {}\n",
    "        self.token_to_idx = token_to_idx\n",
    "\n",
    "        # Index to token\n",
    "        self.idx_to_token = {idx: token \\\n",
    "                             for token, idx in self.token_to_idx.items()}\n",
    "\n",
    "    def to_serializable(self):\n",
    "        return {'token_to_idx': self.token_to_idx}\n",
    "\n",
    "    @classmethod\n",
    "    def from_serializable(cls, contents):\n",
    "        return cls(**contents)\n",
    "\n",
    "    def add_token(self, token):\n",
    "        if token in self.token_to_idx:\n",
    "            index = self.token_to_idx[token]\n",
    "        else:\n",
    "            index = len(self.token_to_idx)\n",
    "            self.token_to_idx[token] = index\n",
    "            self.idx_to_token[index] = token\n",
    "        return index\n",
    "\n",
    "    def add_tokens(self, tokens):\n",
    "        return [self.add_token[token] for token in tokens]\n",
    "\n",
    "    def lookup_token(self, token):\n",
    "        return self.token_to_idx[token]\n",
    "\n",
    "    def lookup_index(self, index):\n",
    "        if index not in self.idx_to_token:\n",
    "            raise KeyError(\"the index (%d) is not in the Vocabulary\" % index)\n",
    "        return self.idx_to_token[index]\n",
    "\n",
    "    def __str__(self):\n",
    "        return \"<Vocabulary(size=%d)>\" % len(self)\n",
    "\n",
    "    def __len__(self):\n",
    "        return len(self.token_to_idx)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 85
    },
    "colab_type": "code",
    "id": "WKnfFKY95v81",
    "outputId": "1835ff66-6548-424e-bdfc-94b0c89d9896"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<Vocabulary(size=4)>\n",
      "4\n",
      "0\n",
      "Business\n"
     ]
    }
   ],
   "source": [
    "# Vocabulary instance\n",
    "category_vocab = Vocabulary()\n",
    "for index, row in df.iterrows():\n",
    "    category_vocab.add_token(row.category)\n",
    "print (category_vocab) # __str__\n",
    "print (len(category_vocab)) # __len__\n",
    "index = category_vocab.lookup_token(\"Business\")\n",
    "print (index)\n",
    "print (category_vocab.lookup_index(index))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Th-KJnOi_74q"
   },
   "source": [
    "## Sequence vocabulary"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "jxO9-6vhBCSO"
   },
   "source": [
    "Next, we're going to create our Vocabulary classes for the article's title, which is a sequence of tokens."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "ip8ViI2v_7Y2"
   },
   "outputs": [],
   "source": [
    "from collections import Counter\n",
    "import string"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "S2LTR5NjBDCG"
   },
   "outputs": [],
   "source": [
    "class SequenceVocabulary(Vocabulary):\n",
    "    def __init__(self, token_to_idx=None, unk_token=\"<UNK>\",\n",
    "                 mask_token=\"<MASK>\", begin_seq_token=\"<BEGIN>\",\n",
    "                 end_seq_token=\"<END>\"):\n",
    "\n",
    "        super(SequenceVocabulary, self).__init__(token_to_idx)\n",
    "\n",
    "        self.mask_token = mask_token\n",
    "        self.unk_token = unk_token\n",
    "        self.begin_seq_token = begin_seq_token\n",
    "        self.end_seq_token = end_seq_token\n",
    "\n",
    "        self.mask_index = self.add_token(self.mask_token)\n",
    "        self.unk_index = self.add_token(self.unk_token)\n",
    "        self.begin_seq_index = self.add_token(self.begin_seq_token)\n",
    "        self.end_seq_index = self.add_token(self.end_seq_token)\n",
    "        \n",
    "        # Index to token\n",
    "        self.idx_to_token = {idx: token \\\n",
    "                             for token, idx in self.token_to_idx.items()}\n",
    "\n",
    "    def to_serializable(self):\n",
    "        contents = super(SequenceVocabulary, self).to_serializable()\n",
    "        contents.update({'unk_token': self.unk_token,\n",
    "                         'mask_token': self.mask_token,\n",
    "                         'begin_seq_token': self.begin_seq_token,\n",
    "                         'end_seq_token': self.end_seq_token})\n",
    "        return contents\n",
    "\n",
    "    def lookup_token(self, token):\n",
    "        return self.token_to_idx.get(token, self.unk_index)\n",
    "    \n",
    "    def lookup_index(self, index):\n",
    "        if index not in self.idx_to_token:\n",
    "            raise KeyError(\"the index (%d) is not in the SequenceVocabulary\" % index)\n",
    "        return self.idx_to_token[index]\n",
    "    \n",
    "    def __str__(self):\n",
    "        return \"<SequenceVocabulary(size=%d)>\" % len(self.token_to_idx)\n",
    "\n",
    "    def __len__(self):\n",
    "        return len(self.token_to_idx)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 85
    },
    "colab_type": "code",
    "id": "DvDPVDbsBDE4",
    "outputId": "bce5fc31-b1d7-41d6-e919-1cc72b584e5b"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<SequenceVocabulary(size=4400)>\n",
      "4400\n",
      "4\n",
      "general\n"
     ]
    }
   ],
   "source": [
    "# Get word counts\n",
    "word_counts = Counter()\n",
    "for title in split_df.title:\n",
    "    for token in title.split(\" \"):\n",
    "        if token not in string.punctuation:\n",
    "            word_counts[token] += 1\n",
    "\n",
    "# Create SequenceVocabulary instance\n",
    "title_vocab = SequenceVocabulary()\n",
    "for word, word_count in word_counts.items():\n",
    "    if word_count >= args.cutoff:\n",
    "        title_vocab.add_token(word)\n",
    "print (title_vocab) # __str__\n",
    "print (len(title_vocab)) # __len__\n",
    "index = title_vocab.lookup_token(\"general\")\n",
    "print (index)\n",
    "print (title_vocab.lookup_index(index))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "6TuUHvlI6JGh"
   },
   "source": [
    "## Vectorizer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "s8IsOPHT5v_i"
   },
   "outputs": [],
   "source": [
    "class NewsVectorizer(object):\n",
    "    def __init__(self, title_vocab, category_vocab):\n",
    "        self.title_vocab = title_vocab\n",
    "        self.category_vocab = category_vocab\n",
    "\n",
    "    def vectorize(self, title):\n",
    "        indices = [self.title_vocab.lookup_token(token) for token in title.split(\" \")]\n",
    "        indices = [self.title_vocab.begin_seq_index] + indices + \\\n",
    "            [self.title_vocab.end_seq_index]\n",
    "        \n",
    "        # Create vector\n",
    "        title_length = len(indices)\n",
    "        vector = np.zeros(title_length, dtype=np.int64)\n",
    "        vector[:len(indices)] = indices\n",
    "\n",
    "        return vector\n",
    "    \n",
    "    def unvectorize(self, vector):\n",
    "        tokens = [self.title_vocab.lookup_index(index) for index in vector]\n",
    "        title = \" \".join(token for token in tokens)\n",
    "        return title\n",
    "\n",
    "    @classmethod\n",
    "    def from_dataframe(cls, df, cutoff):\n",
    "        \n",
    "        # Create class vocab\n",
    "        category_vocab = Vocabulary()        \n",
    "        for category in sorted(set(df.category)):\n",
    "            category_vocab.add_token(category)\n",
    "\n",
    "        # Get word counts\n",
    "        word_counts = Counter()\n",
    "        for title in df.title:\n",
    "            for token in title.split(\" \"):\n",
    "                word_counts[token] += 1\n",
    "        \n",
    "        # Create title vocab\n",
    "        title_vocab = SequenceVocabulary()\n",
    "        for word, word_count in word_counts.items():\n",
    "            if word_count >= cutoff:\n",
    "                title_vocab.add_token(word)\n",
    "        \n",
    "        return cls(title_vocab, category_vocab)\n",
    "\n",
    "    @classmethod\n",
    "    def from_serializable(cls, contents):\n",
    "        title_vocab = SequenceVocabulary.from_serializable(contents['title_vocab'])\n",
    "        category_vocab = Vocabulary.from_serializable(contents['category_vocab'])\n",
    "        return cls(title_vocab=title_vocab, category_vocab=category_vocab)\n",
    "    \n",
    "    def to_serializable(self):\n",
    "        return {'title_vocab': self.title_vocab.to_serializable(),\n",
    "                'category_vocab': self.category_vocab.to_serializable()}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 102
    },
    "colab_type": "code",
    "id": "p2blo9sT5wCS",
    "outputId": "5f43dd85-25ed-4851-e699-e667a06f5d3f"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<SequenceVocabulary(size=4405)>\n",
      "<Vocabulary(size=4)>\n",
      "(11,)\n",
      "[   2    1 4152 1232   25    1 2393 4077   39   31    3]\n",
      "<BEGIN> <UNK> federer wins the <UNK> tennis tournament .  <END>\n"
     ]
    }
   ],
   "source": [
    "# Vectorizer instance\n",
    "vectorizer = NewsVectorizer.from_dataframe(split_df, cutoff=args.cutoff)\n",
    "print (vectorizer.title_vocab)\n",
    "print (vectorizer.category_vocab)\n",
    "vectorized_title = vectorizer.vectorize(preprocess_text(\n",
    "    \"Roger Federer wins the Wimbledon tennis tournament.\"))\n",
    "print (np.shape(vectorized_title))\n",
    "print (vectorized_title)\n",
    "print (vectorizer.unvectorize(vectorized_title))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "jTpYV1FkHBTi"
   },
   "source": [
    "## Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "_PcKZGbYIVA7"
   },
   "outputs": [],
   "source": [
    "from torch.utils.data import Dataset, DataLoader"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Lrssjuxb1NZy"
   },
   "outputs": [],
   "source": [
    "class NewsDataset(Dataset):\n",
    "    def __init__(self, df, vectorizer):\n",
    "        self.df = df\n",
    "        self.vectorizer = vectorizer\n",
    "        \n",
    "        # Max title length\n",
    "        get_length = lambda title: len(title.split(\" \"))\n",
    "        self.max_seq_length = max(map(get_length, df.title)) + 2 # (<BEGIN> + <END>)\n",
    "\n",
    "        # Data splits\n",
    "        self.train_df = self.df[self.df.split=='train']\n",
    "        self.train_size = len(self.train_df)\n",
    "        self.val_df = self.df[self.df.split=='val']\n",
    "        self.val_size = len(self.val_df)\n",
    "        self.test_df = self.df[self.df.split=='test']\n",
    "        self.test_size = len(self.test_df)\n",
    "        self.lookup_dict = {'train': (self.train_df, self.train_size), \n",
    "                            'val': (self.val_df, self.val_size),\n",
    "                            'test': (self.test_df, self.test_size)}\n",
    "        self.set_split('train')\n",
    "\n",
    "        # Class weights (for imbalances)\n",
    "        class_counts = df.category.value_counts().to_dict()\n",
    "        def sort_key(item):\n",
    "            return self.vectorizer.category_vocab.lookup_token(item[0])\n",
    "        sorted_counts = sorted(class_counts.items(), key=sort_key)\n",
    "        frequencies = [count for _, count in sorted_counts]\n",
    "        self.class_weights = 1.0 / torch.tensor(frequencies, dtype=torch.float32)\n",
    "\n",
    "    @classmethod\n",
    "    def load_dataset_and_make_vectorizer(cls, df, cutoff):\n",
    "        train_df = df[df.split=='train']\n",
    "        return cls(df, NewsVectorizer.from_dataframe(train_df, cutoff))\n",
    "\n",
    "    @classmethod\n",
    "    def load_dataset_and_load_vectorizer(cls, df, vectorizer_filepath):\n",
    "        vectorizer = cls.load_vectorizer_only(vectorizer_filepath)\n",
    "        return cls(df, vectorizer)\n",
    "\n",
    "    def load_vectorizer_only(vectorizer_filepath):\n",
    "        with open(vectorizer_filepath) as fp:\n",
    "            return NewsVectorizer.from_serializable(json.load(fp))\n",
    "\n",
    "    def save_vectorizer(self, vectorizer_filepath):\n",
    "        with open(vectorizer_filepath, \"w\") as fp:\n",
    "            json.dump(self.vectorizer.to_serializable(), fp)\n",
    "\n",
    "    def set_split(self, split=\"train\"):\n",
    "        self.target_split = split\n",
    "        self.target_df, self.target_size = self.lookup_dict[split]\n",
    "\n",
    "    def __str__(self):\n",
    "        return \"<Dataset(split={0}, size={1})\".format(\n",
    "            self.target_split, self.target_size)\n",
    "\n",
    "    def __len__(self):\n",
    "        return self.target_size\n",
    "\n",
    "    def __getitem__(self, index):\n",
    "        row = self.target_df.iloc[index]\n",
    "        title_vector = self.vectorizer.vectorize(row.title)\n",
    "        category_index = self.vectorizer.category_vocab.lookup_token(row.category)\n",
    "        return {'title': title_vector, 'category': category_index}\n",
    "\n",
    "    def get_num_batches(self, batch_size):\n",
    "        return len(self) // batch_size\n",
    "\n",
    "    def generate_batches(self, batch_size, collate_fn, shuffle=True, \n",
    "                         drop_last=False, device=\"cpu\"):\n",
    "        dataloader = DataLoader(dataset=self, batch_size=batch_size,\n",
    "                                collate_fn=collate_fn, shuffle=shuffle, \n",
    "                                drop_last=drop_last)\n",
    "        for data_dict in dataloader:\n",
    "            out_data_dict = {}\n",
    "            for name, tensor in data_dict.items():\n",
    "                out_data_dict[name] = data_dict[name].to(device)\n",
    "            yield out_data_dict"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 85
    },
    "colab_type": "code",
    "id": "e9I-AFKsIP2e",
    "outputId": "44c90229-bff4-40a6-977e-6399c793254f"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<Dataset(split=train, size=84000)\n",
      "[ 2 32 33 10 34 13  3]\n",
      "<BEGIN> software firm to cut jobs <END>\n",
      "tensor([3.3333e-05, 3.3333e-05, 3.3333e-05, 3.3333e-05])\n"
     ]
    }
   ],
   "source": [
    "# Dataset instance\n",
    "dataset = NewsDataset.load_dataset_and_make_vectorizer(df=split_df, \n",
    "                                                       cutoff=args.cutoff)\n",
    "print (dataset) # __str__\n",
    "title_vector = dataset[5]['title'] # __getitem__\n",
    "print (title_vector)\n",
    "print (dataset.vectorizer.unvectorize(title_vector))\n",
    "print (dataset.class_weights)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "IxjCMr_DINGk"
   },
   "source": [
    "## Model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "202KnN2lLj61"
   },
   "source": [
    "input → embedding → conv → FC \n",
    "\n",
    "We will be using 1d conv operations ([nn.Conv1D](https://pytorch.org/docs/stable/nn.html#torch.nn.Conv1d)) even though our inputs are words because we are not representing them at a character level. The inputs are of shape $\\in \\mathbb{R}^{NXSXE}$\n",
    "* where:\n",
    "    * N = batchsize\n",
    "    * S = max sentence length \n",
    "    * E = embedding dim at a word level"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "j-Xgp0F3INRA"
   },
   "outputs": [],
   "source": [
    "import torch.nn as nn\n",
    "import torch.nn.functional as F"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Lfw7csveHApA"
   },
   "outputs": [],
   "source": [
    "class NewsModel(nn.Module):\n",
    "    def __init__(self, embedding_dim, num_embeddings, num_input_channels, \n",
    "                 num_channels, hidden_dim, num_classes, dropout_p, \n",
    "                 pretrained_embeddings=None, freeze_embeddings=False,\n",
    "                 padding_idx=0):\n",
    "        super(NewsModel, self).__init__()\n",
    "        \n",
    "        if pretrained_embeddings is None:\n",
    "            self.embeddings = nn.Embedding(embedding_dim=embedding_dim,\n",
    "                                          num_embeddings=num_embeddings,\n",
    "                                          padding_idx=padding_idx)\n",
    "        else:\n",
    "            pretrained_embeddings = torch.from_numpy(pretrained_embeddings).float()\n",
    "            self.embeddings = nn.Embedding(embedding_dim=embedding_dim,\n",
    "                                           num_embeddings=num_embeddings,\n",
    "                                           padding_idx=padding_idx,\n",
    "                                           _weight=pretrained_embeddings)\n",
    "        \n",
    "        # Conv weights\n",
    "        self.conv = nn.ModuleList([nn.Conv1d(num_input_channels, num_channels, \n",
    "                                             kernel_size=f) for f in [2,3,4]])\n",
    "     \n",
    "        # FC weights\n",
    "        self.dropout = nn.Dropout(dropout_p)\n",
    "        self.fc1 = nn.Linear(num_channels*3, hidden_dim)\n",
    "        self.fc2 = nn.Linear(hidden_dim, num_classes)\n",
    "        \n",
    "        if freeze_embeddings:\n",
    "            self.embeddings.weight.requires_grad = False\n",
    "\n",
    "    def forward(self, x_in, channel_first=False, apply_softmax=False):\n",
    "        \n",
    "        # Embed\n",
    "        x_in = self.embeddings(x_in)\n",
    "\n",
    "        # Rearrange input so num_channels is in dim 1 (N, C, L)\n",
    "        if not channel_first:\n",
    "            x_in = x_in.transpose(1, 2)\n",
    "            \n",
    "        # Conv outputs\n",
    "        z1 = self.conv[0](x_in)\n",
    "        z1 = F.max_pool1d(z1, z1.size(2)).squeeze(2)\n",
    "        z2 = self.conv[1](x_in)\n",
    "        z2 = F.max_pool1d(z2, z2.size(2)).squeeze(2)\n",
    "        z3 = self.conv[2](x_in)\n",
    "        z3 = F.max_pool1d(z3, z3.size(2)).squeeze(2)\n",
    "        \n",
    "        # Concat conv outputs\n",
    "        z = torch.cat([z1, z2, z3], 1)\n",
    "\n",
    "        # FC layers\n",
    "        z = self.dropout(z)\n",
    "        z = self.fc1(z)\n",
    "        y_pred = self.fc2(z)\n",
    "        \n",
    "        if apply_softmax:\n",
    "            y_pred = F.softmax(y_pred, dim=1)\n",
    "        return y_pred"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "d7_4SiLkKji8"
   },
   "source": [
    "## Training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "UQAIfKN2HAtN"
   },
   "outputs": [],
   "source": [
    "import torch.optim as optim"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "RwgZASWYHAwB"
   },
   "outputs": [],
   "source": [
    "class Trainer(object):\n",
    "    def __init__(self, dataset, model, model_state_file, save_dir, device, shuffle, \n",
    "               num_epochs, batch_size, learning_rate, early_stopping_criteria):\n",
    "        self.dataset = dataset\n",
    "        self.class_weights = dataset.class_weights.to(device)\n",
    "        self.model = model.to(device)\n",
    "        self.save_dir = save_dir\n",
    "        self.device = device\n",
    "        self.shuffle = shuffle\n",
    "        self.num_epochs = num_epochs\n",
    "        self.batch_size = batch_size\n",
    "        self.loss_func = nn.CrossEntropyLoss(self.class_weights)\n",
    "        self.optimizer = optim.Adam(self.model.parameters(), lr=learning_rate)\n",
    "        self.scheduler = optim.lr_scheduler.ReduceLROnPlateau(\n",
    "            optimizer=self.optimizer, mode='min', factor=0.5, patience=1)\n",
    "        self.train_state = {\n",
    "            'done_training': False,\n",
    "            'stop_early': False, \n",
    "            'early_stopping_step': 0,\n",
    "            'early_stopping_best_val': 1e8,\n",
    "            'early_stopping_criteria': early_stopping_criteria,\n",
    "            'learning_rate': learning_rate,\n",
    "            'epoch_index': 0,\n",
    "            'train_loss': [],\n",
    "            'train_acc': [],\n",
    "            'val_loss': [],\n",
    "            'val_acc': [],\n",
    "            'test_loss': -1,\n",
    "            'test_acc': -1,\n",
    "            'model_filename': model_state_file}\n",
    "    \n",
    "    def update_train_state(self):\n",
    "\n",
    "        # Verbose\n",
    "        print (\"[EPOCH]: {0} | [LR]: {1} | [TRAIN LOSS]: {2:.2f} | [TRAIN ACC]: {3:.1f}% | [VAL LOSS]: {4:.2f} | [VAL ACC]: {5:.1f}%\".format(\n",
    "          self.train_state['epoch_index'], self.train_state['learning_rate'], \n",
    "            self.train_state['train_loss'][-1], self.train_state['train_acc'][-1], \n",
    "            self.train_state['val_loss'][-1], self.train_state['val_acc'][-1]))\n",
    "\n",
    "        # Save one model at least\n",
    "        if self.train_state['epoch_index'] == 0:\n",
    "            torch.save(self.model.state_dict(), self.train_state['model_filename'])\n",
    "            self.train_state['stop_early'] = False\n",
    "\n",
    "        # Save model if performance improved\n",
    "        elif self.train_state['epoch_index'] >= 1:\n",
    "            loss_tm1, loss_t = self.train_state['val_loss'][-2:]\n",
    "\n",
    "            # If loss worsened\n",
    "            if loss_t >= self.train_state['early_stopping_best_val']:\n",
    "                # Update step\n",
    "                self.train_state['early_stopping_step'] += 1\n",
    "\n",
    "            # Loss decreased\n",
    "            else:\n",
    "                # Save the best model\n",
    "                if loss_t < self.train_state['early_stopping_best_val']:\n",
    "                    torch.save(self.model.state_dict(), self.train_state['model_filename'])\n",
    "\n",
    "                # Reset early stopping step\n",
    "                self.train_state['early_stopping_step'] = 0\n",
    "\n",
    "            # Stop early ?\n",
    "            self.train_state['stop_early'] = self.train_state['early_stopping_step'] \\\n",
    "              >= self.train_state['early_stopping_criteria']\n",
    "        return self.train_state\n",
    "  \n",
    "    def compute_accuracy(self, y_pred, y_target):\n",
    "        _, y_pred_indices = y_pred.max(dim=1)\n",
    "        n_correct = torch.eq(y_pred_indices, y_target).sum().item()\n",
    "        return n_correct / len(y_pred_indices) * 100\n",
    "    \n",
    "    def pad_seq(self, seq, length):\n",
    "        vector = np.zeros(length, dtype=np.int64)\n",
    "        vector[:len(seq)] = seq\n",
    "        vector[len(seq):] = self.dataset.vectorizer.title_vocab.mask_index\n",
    "        return vector\n",
    "    \n",
    "    def collate_fn(self, batch):\n",
    "        \n",
    "        # Make a deep copy\n",
    "        batch_copy = copy.deepcopy(batch)\n",
    "        processed_batch = {\"title\": [], \"category\": []}\n",
    "        \n",
    "        # Get max sequence length\n",
    "        max_seq_len = max([len(sample[\"title\"]) for sample in batch_copy])\n",
    "        \n",
    "        # Pad\n",
    "        for i, sample in enumerate(batch_copy):\n",
    "            seq = sample[\"title\"]\n",
    "            category = sample[\"category\"]\n",
    "            padded_seq = self.pad_seq(seq, max_seq_len)\n",
    "            processed_batch[\"title\"].append(padded_seq)\n",
    "            processed_batch[\"category\"].append(category)\n",
    "            \n",
    "        # Convert to appropriate tensor types\n",
    "        processed_batch[\"title\"] = torch.LongTensor(\n",
    "            processed_batch[\"title\"])\n",
    "        processed_batch[\"category\"] = torch.LongTensor(\n",
    "            processed_batch[\"category\"])\n",
    "        \n",
    "        return processed_batch    \n",
    "  \n",
    "    def run_train_loop(self):\n",
    "        for epoch_index in range(self.num_epochs):\n",
    "            self.train_state['epoch_index'] = epoch_index\n",
    "      \n",
    "            # Iterate over train dataset\n",
    "\n",
    "            # initialize batch generator, set loss and acc to 0, set train mode on\n",
    "            self.dataset.set_split('train')\n",
    "            batch_generator = self.dataset.generate_batches(\n",
    "                batch_size=self.batch_size, collate_fn=self.collate_fn, \n",
    "                shuffle=self.shuffle, device=self.device)\n",
    "            running_loss = 0.0\n",
    "            running_acc = 0.0\n",
    "            self.model.train()\n",
    "\n",
    "            for batch_index, batch_dict in enumerate(batch_generator):\n",
    "                # zero the gradients\n",
    "                self.optimizer.zero_grad()\n",
    "\n",
    "                # compute the output\n",
    "                y_pred = self.model(batch_dict['title'])\n",
    "\n",
    "                # compute the loss\n",
    "                loss = self.loss_func(y_pred, batch_dict['category'])\n",
    "                loss_t = loss.item()\n",
    "                running_loss += (loss_t - running_loss) / (batch_index + 1)\n",
    "\n",
    "                # compute gradients using loss\n",
    "                loss.backward()\n",
    "\n",
    "                # use optimizer to take a gradient step\n",
    "                self.optimizer.step()\n",
    "                \n",
    "                # compute the accuracy\n",
    "                acc_t = self.compute_accuracy(y_pred, batch_dict['category'])\n",
    "                running_acc += (acc_t - running_acc) / (batch_index + 1)\n",
    "\n",
    "            self.train_state['train_loss'].append(running_loss)\n",
    "            self.train_state['train_acc'].append(running_acc)\n",
    "\n",
    "            # Iterate over val dataset\n",
    "\n",
    "            # initialize batch generator, set loss and acc to 0; set eval mode on\n",
    "            self.dataset.set_split('val')\n",
    "            batch_generator = self.dataset.generate_batches(\n",
    "                batch_size=self.batch_size, collate_fn=self.collate_fn, \n",
    "                shuffle=self.shuffle, device=self.device)\n",
    "            running_loss = 0.\n",
    "            running_acc = 0.\n",
    "            self.model.eval()\n",
    "\n",
    "            for batch_index, batch_dict in enumerate(batch_generator):\n",
    "\n",
    "                # compute the output\n",
    "                y_pred =  self.model(batch_dict['title'])\n",
    "\n",
    "                # compute the loss\n",
    "                loss = self.loss_func(y_pred, batch_dict['category'])\n",
    "                loss_t = loss.to(\"cpu\").item()\n",
    "                running_loss += (loss_t - running_loss) / (batch_index + 1)\n",
    "\n",
    "                # compute the accuracy\n",
    "                acc_t = self.compute_accuracy(y_pred, batch_dict['category'])\n",
    "                running_acc += (acc_t - running_acc) / (batch_index + 1)\n",
    "\n",
    "            self.train_state['val_loss'].append(running_loss)\n",
    "            self.train_state['val_acc'].append(running_acc)\n",
    "\n",
    "            self.train_state = self.update_train_state()\n",
    "            self.scheduler.step(self.train_state['val_loss'][-1])\n",
    "            if self.train_state['stop_early']:\n",
    "                break\n",
    "          \n",
    "    def run_test_loop(self):\n",
    "        # initialize batch generator, set loss and acc to 0; set eval mode on\n",
    "        self.dataset.set_split('test')\n",
    "        batch_generator = self.dataset.generate_batches(\n",
    "                batch_size=self.batch_size, collate_fn=self.collate_fn, \n",
    "                shuffle=self.shuffle, device=self.device)\n",
    "        running_loss = 0.0\n",
    "        running_acc = 0.0\n",
    "        self.model.eval()\n",
    "\n",
    "        for batch_index, batch_dict in enumerate(batch_generator):\n",
    "            # compute the output\n",
    "            y_pred =  self.model(batch_dict['title'])\n",
    "\n",
    "            # compute the loss\n",
    "            loss = self.loss_func(y_pred, batch_dict['category'])\n",
    "            loss_t = loss.item()\n",
    "            running_loss += (loss_t - running_loss) / (batch_index + 1)\n",
    "\n",
    "            # compute the accuracy\n",
    "            acc_t = self.compute_accuracy(y_pred, batch_dict['category'])\n",
    "            running_acc += (acc_t - running_acc) / (batch_index + 1)\n",
    "\n",
    "        self.train_state['test_loss'] = running_loss\n",
    "        self.train_state['test_acc'] = running_acc\n",
    "    \n",
    "    def plot_performance(self):\n",
    "        # Figure size\n",
    "        plt.figure(figsize=(15,5))\n",
    "\n",
    "        # Plot Loss\n",
    "        plt.subplot(1, 2, 1)\n",
    "        plt.title(\"Loss\")\n",
    "        plt.plot(trainer.train_state[\"train_loss\"], label=\"train\")\n",
    "        plt.plot(trainer.train_state[\"val_loss\"], label=\"val\")\n",
    "        plt.legend(loc='upper right')\n",
    "\n",
    "        # Plot Accuracy\n",
    "        plt.subplot(1, 2, 2)\n",
    "        plt.title(\"Accuracy\")\n",
    "        plt.plot(trainer.train_state[\"train_acc\"], label=\"train\")\n",
    "        plt.plot(trainer.train_state[\"val_acc\"], label=\"val\")\n",
    "        plt.legend(loc='lower right')\n",
    "\n",
    "        # Save figure\n",
    "        plt.savefig(os.path.join(self.save_dir, \"performance.png\"))\n",
    "\n",
    "        # Show plots\n",
    "        plt.show()\n",
    "    \n",
    "    def save_train_state(self):\n",
    "        self.train_state[\"done_training\"] = True\n",
    "        with open(os.path.join(self.save_dir, \"train_state.json\"), \"w\") as fp:\n",
    "            json.dump(self.train_state, fp)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 204
    },
    "colab_type": "code",
    "id": "LJfKGc9cHA0y",
    "outputId": "ed3bbe45-3c58-4ebe-db4d-8efbf1b326d1"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<bound method Module.named_modules of NewsModel(\n",
      "  (embeddings): Embedding(3407, 100, padding_idx=0)\n",
      "  (conv): ModuleList(\n",
      "    (0): Conv1d(100, 100, kernel_size=(2,), stride=(1,))\n",
      "    (1): Conv1d(100, 100, kernel_size=(3,), stride=(1,))\n",
      "    (2): Conv1d(100, 100, kernel_size=(4,), stride=(1,))\n",
      "  )\n",
      "  (dropout): Dropout(p=0.1)\n",
      "  (fc1): Linear(in_features=300, out_features=100, bias=True)\n",
      "  (fc2): Linear(in_features=100, out_features=4, bias=True)\n",
      ")>\n"
     ]
    }
   ],
   "source": [
    "# Initialization\n",
    "dataset = NewsDataset.load_dataset_and_make_vectorizer(df=split_df, \n",
    "                                                       cutoff=args.cutoff)\n",
    "dataset.save_vectorizer(args.vectorizer_file)\n",
    "vectorizer = dataset.vectorizer\n",
    "model = NewsModel(embedding_dim=args.embedding_dim, \n",
    "                  num_embeddings=len(vectorizer.title_vocab), \n",
    "                  num_input_channels=args.embedding_dim, \n",
    "                  num_channels=args.num_filters, hidden_dim=args.hidden_dim, \n",
    "                  num_classes=len(vectorizer.category_vocab), \n",
    "                  dropout_p=args.dropout_p, pretrained_embeddings=None, \n",
    "                  padding_idx=vectorizer.title_vocab.mask_index)\n",
    "print (model.named_modules)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 102
    },
    "colab_type": "code",
    "id": "wDxPyGIVKoUK",
    "outputId": "774c981e-6728-46cd-e1a6-3a8ee5d29a79"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[EPOCH]: 0 | [LR]: 0.001 | [TRAIN LOSS]: 0.79 | [TRAIN ACC]: 69.1% | [VAL LOSS]: 0.59 | [VAL ACC]: 78.5%\n",
      "[EPOCH]: 1 | [LR]: 0.001 | [TRAIN LOSS]: 0.50 | [TRAIN ACC]: 81.6% | [VAL LOSS]: 0.51 | [VAL ACC]: 81.8%\n",
      "[EPOCH]: 2 | [LR]: 0.001 | [TRAIN LOSS]: 0.42 | [TRAIN ACC]: 84.8% | [VAL LOSS]: 0.48 | [VAL ACC]: 82.7%\n",
      "[EPOCH]: 3 | [LR]: 0.001 | [TRAIN LOSS]: 0.37 | [TRAIN ACC]: 86.7% | [VAL LOSS]: 0.50 | [VAL ACC]: 82.8%\n",
      "[EPOCH]: 4 | [LR]: 0.001 | [TRAIN LOSS]: 0.32 | [TRAIN ACC]: 88.2% | [VAL LOSS]: 0.50 | [VAL ACC]: 82.4%\n"
     ]
    }
   ],
   "source": [
    "# Train\n",
    "trainer = Trainer(dataset=dataset, model=model, \n",
    "                  model_state_file=args.model_state_file, \n",
    "                  save_dir=args.save_dir, device=args.device,\n",
    "                  shuffle=args.shuffle, num_epochs=args.num_epochs, \n",
    "                  batch_size=args.batch_size, learning_rate=args.learning_rate, \n",
    "                  early_stopping_criteria=args.early_stopping_criteria)\n",
    "trainer.run_train_loop()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 335
    },
    "colab_type": "code",
    "id": "V9Ruxpc8WT17",
    "outputId": "63933b69-9e07-45f1-82d2-52eab6db9c9a"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA2gAAAE+CAYAAAD4XjP+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3Xlc1Peh//vXLOzrsArILop73PcN\nNLil2ZpG25i0Sc+5TdOex+80zTWX9tQ0TZr03trTJD1Nm56cpu1JE9sUY9wTVIxGXHHfRTZBYNhl\nZ2DuHyhK1bgBw8D7+XgYZr7LzHtGA7zn+/l+vga73W5HREREREREHM7o6AAiIiIiIiLSTgVNRERE\nRESkl1BBExERERER6SVU0ERERERERHoJFTQREREREZFeQgVNRERERESkl1BBE7lLQ4YMobi42NEx\nREREesSSJUv4yle+4ugYIn2eCpqIiIiIfKkzZ87g4+NDeHg4Bw8edHQckT5NBU2kizU1NfGTn/yE\nlJQUFixYwOuvv05raysA//u//8uCBQuYP38+X/3qVzl79uyXLhcREekNVq9ezfz581m8eDEff/xx\nx/KPP/6YlJQUUlJSeOGFF2hubr7p8j179jBv3ryOfa+9/9Zbb/HjH/+Yr371q7z33nu0tbXx05/+\nlJSUFJKSknjhhRdoaWkBoKKigu985zskJyfzwAMPsHPnTjIyMli8eHGnzI888gjp6end/daIdDmz\nowOI9DV/+tOfKC4uZv369dhsNp544gnWrVtHcnIyb7zxBtu2bcPb25uNGzeSkZFBWFjYDZcnJCQ4\n+qWIiIjQ2trKZ599xnPPPYfJZGLlypU0NzdTWlrKL37xCz7++GNCQkL4/ve/z5///Gfmz59/w+Uj\nR4780ufZvn07a9asISAggM2bN7N//37WrVtHW1sbDz/8MBs2bODBBx9k5cqVxMfH87vf/Y4TJ07w\nrW99ix07dmC1Wjl16hSJiYkUFRWRn5/PzJkze+hdEuk6KmgiXSwjI4Onn34as9mM2WzmgQce4Isv\nvmDhwoUYDAY++ugjFi9ezIIFCwBoaWm54XIREZHeYOfOnYwcORJvb28AJk6cyLZt26iqqmLMmDGE\nhoYCsHLlSkwmE//4xz9uuPzAgQNf+jyjR48mICAAgJSUFObMmYOLiwsAI0eOpKCgAGgvcn/4wx8A\nGDZsGFu2bMHV1ZWUlBTWr19PYmIi6enpJCcn4+rq2vVviEg30xBHkS5WUVGBn59fx30/Pz/Ky8tx\ncXHhvffeIysri5SUFL7+9a9z+vTpmy4XERHpDdLS0sjIyGD8+PGMHz+eTz/9lNWrV1NZWYmvr2/H\ndm5ubpjN5psuv5Vrf3ZWVFSwfPlyUlJSmD9/Plu2bMFutwNQVVWFj49Px7ZXiuOiRYtYv349AOnp\n6SxcuPDeXriIg6igiXSxoKAgqqqqOu5XVVURFBQEtH/S9+abb5KZmcn06dNZsWLFly4XERFxpOrq\navbu3cuePXvYv38/+/fvZ9++fRw9ehSj0UhlZWXHtrW1tZSVlWGxWG643GQydZyTDVBTU3PT5/3P\n//xPzGYza9euZdOmTcyaNatjnb+/f6fHv3DhAi0tLUyYMAGbzca2bds4e/YsU6dO7aq3QaRHqaCJ\ndLHZs2fz0Ucf0draSn19PWvWrGHWrFmcPn2af/u3f6O5uRlXV1dGjBiBwWC46XIRERFHW79+PZMn\nT+40VNBsNjN9+nSam5vJysriwoUL2O12VqxYwUcffcSsWbNuuDw4OBir1Up5eTmtra2sXbv2ps9b\nXl7O4MGDcXV15dSpUxw8eJD6+noAkpKSWL16NQDnzp3jkUceobW1FaPRyMKFC/nZz35GUlJSx/BI\nEWejc9BE7sGyZcswmUwd91955RWWLVtGQUEBixYtwmAwMH/+/I7zygYOHMjixYtxcXHBy8uLn/zk\nJwwePPiGy0VERBzt448/5qmnnrpu+bx58/jtb3/Lyy+/zFNPPYXJZGLkyJF861vfws3N7abLH330\nUR566CHCw8N58MEHOXny5A2f9+mnn2b58uWkpaUxfvx4li9fzo9+9CNGjRrFCy+8wPLly0lKSsLL\ny4tf/vKXuLu7A+3DHP/4xz9qeKM4NYP9yoBeEREREREnVlZWxsMPP0xGRkanD1BFnImGOIqIiIhI\nn/Dmm2+ydOlSlTNxaipoIiIiIuLUysrKSE5OpqysjKefftrRcUTuiYY4ioiIiIiI9BI6giYiIiIi\nItJLqKCJiIiIiIj0Ej0+zb7VeumeH8Ni8aSysr4L0vQMZ8qrrN3DmbKCc+VV1u7RVVmDg326IE3/\n0d9+RjpTVnCuvMraPZS1+zhT3q7I+mU/H2+roP385z/n8OHDGAwGUlNTGTVqVMe6999/n08++QSj\n0ciIESP40Y9+dE9hb4fZ7Fwz8zhTXmXtHs6UFZwrr7J2D2fKKp0509+dM2UF58qrrN1DWbuPM+Xt\n7qy3HOK4d+9e8vLyWLVqFa+++iqvvvpqx7ra2lreffdd3n//fT744AOys7M5dOhQtwYWERERERHp\nq25Z0DIzM5k7dy4A8fHxVFdXU1tbC4CLiwsuLi7U19djs9loaGjAz8+vexOLiIiIiIj0UbcsaGVl\nZVgslo77AQEBWK1WANzc3HjuueeYO3cuc+bMYfTo0cTGxnZfWhERERERkT7sjicJufayabW1tfz+\n979n06ZNeHt789RTT3Hq1CkSExNvur/F4tkl4zad7cRzZ8qrrN3DmbKCc+VV1u7hTFlFRET6ilsW\ntJCQEMrKyjrul5aWEhwcDEB2djaRkZEEBAQAMH78eI4dO/alBa2rZgXripmueooz5VXW7uFMWcG5\n8ipr9+iqrCp5IiIid+aWQxynTZvG5s2bATh+/DghISF4e3sDEBERQXZ2No2NjQAcO3aMmJiY7ksr\nIiIiIiLSh93yCNrYsWMZPnw4S5YswWAwsGLFCtLS0vDx8WHevHk888wzPPnkk5hMJsaMGcP48eN7\nIreIiIiIiEifc1vnoP3whz/sdP/aIYxLlixhyZIlXZtKRES6RUbGFmbPTr7ldq+++iqLFz9KeHhE\nD6QSERGRK245xFFERPqGixeLSE/ffFvb/uhHP1I5ExERcYA7nsXR0ZqaW1m74zz3xVlwd3W6+CIi\nDvOrX/2CkyePM2PGBO6/fwEXLxbx61//ltdeexmrtZSGhgaefvpfmTZtBsuWLeN73/sB27Ztoa6u\nlvz8PAoLL/Bv//Y8U6ZMc/RLERER6VEttlZyiy+Rc/ESE0aEYfHovh7idA3ndEEl73x8lDljIliW\nMsTRcUREnMbSpctIS/sbsbHx5Ofn8tvf/jeVlRVMnDiZBQsWU1h4gf/4jxeZNm1Gp/1KS0v45S/f\nZPfuXaxZ8w8VNBER6fMqahrJLqrh3IVqsouqySu+RGtb++XG8q21fHvh0G57bqcraMNiAhgY4k3G\noULmjIlgYIi3oyOJiNyxv209x75TpV36mBMSQ/ha0qDb2nbo0OEA+Pj4cvLkcT75JA2DwUhNTfV1\n244adR/QftmV2trargssIiLSC9ha28gvqSW7sJpzhe2FrKKmqWO9yWggKtSb+HA/Bg30Y87EaOou\nNXZbHqcraGaTkW8/OIKX/rCbv6af4YWlYzAYDI6OJSLiVFxcXAD47LNN1NTU8F//9d/U1NTw7W8v\nu25bk8nUcdtut/dYRhERke5QXddMdmF1RyHLLb5Ei62tY72vpwtjEoIYFOFHfIQfMQN8cHW5+rPQ\n091FBe2fjUsMZVR8IEeyy8k6U8a4IcGOjiQicke+ljToto92dRWj0Uhra2unZVVVVYSFhWM0Gtm+\nfSstLS09mklERKQ7tba1caG0juyiq4XMWnW1XBkMEBnsTXyE3+VC5kuwv4dDDwA5ZUEDeDxpEMdz\nKli19Syj4gNwMZtuvZOISD8WHR3L6dOnCAsLx9/fH4DZs5N48cUfcOLEMRYt+gohISH88Y9/cHBS\nERGRu1Pb0NJ+dKyomnMXqsm5eImmlqsfTnq5mxkVH0h8uC+DIvyICfPFw613VaLeleYOhAV6kTxu\nIJ/uK+DTfQUsmhLj6EgiIr2axWIhLW19p2VhYeH86U8fdty///4FAAQH+2C1XiIu7upRvri4Qfzm\nN+/0TNhepq6ujuXLl1NdXU1LSwvPPfcc77xz9b0oLS3l4Ycf5jvf+U7Hsrfeeou1a9cSGhoKwFe+\n8hUee+yxHs8uItJXtdntXCyraz9vrLCGc4XVFFfUd9omPMiLQRG+HeePhQZ4Yuzlp0c5bUED+Mq0\nGHYdK2bdrjymjgjD4uPm6EgiItIHrV69mtjYWJ5//nlKSkp46qmn2LRpU8f6b3/72zz44IPX7ffk\nk0/yxBNP9GRUEZE+q6HJxvmimsuFrJrsohoammwd691dTQyLsXScOxYX7ouXu4sDE98dpy5onu4u\nPDIrjj9vOk3a9myeWTzM0ZFERKQPslgsnD59GoCamhosFkvHul27dhETE0NYWJij4omI9Dl2u52S\nyoarMysWVlNorePaqapCLR6dJvOICPLCaOzdR8duh1MXNICZo8LZllXIF8eKSRo3kNgwX0dHEhGR\nPmbRokWkpaUxb948ampq+P3vf9+x7s9//jOpqak33G/Tpk1s2bIFV1dXfvzjHxMZGdlTkUVEnEpj\nk41TeZUd545lF9VQ23B14ipXs5HBkf4MGuhHfLgfcRG++Hq6OjBx93H6gmY0Gvj63AR+8deD/PWz\nM6QuG6dp90VEpEutWbOG8PBw3n33XU6dOkVqaippaWmUlJRQX19PVFTUdfvMmjWLyZMnM2HCBNav\nX88rr7zSqdjdiMXiibkLJr0KDva558foKc6UFZwrr7J2D2W9d3a7nZKKek7lVXIqt4JTeRXkFNXQ\n1nb1+FhIgCdjE0MYGhNAYnQAMeG+mE1GB6burDvfW6cvaABDoiyMHxLM/tNWdp8oYcrwAY6OJCIi\nfUhWVhbTp08HIDExkdLSUlpbW9m+fTuTJ0++4T6jRo3quJ2UlMQvf/nLWz5PZWX9Lbe5lSsTvDgD\nZ8oKzpVXWbuHst6dFlsrucWXOibyyC6sprquuWO92WRgSJSFqBCvjuGK/t6d55aorKjr6dg31RXv\n7ZcVvD5R0AC+NmcQh86V81FGNmMTgnFz1bT7IiLSNaKjozl8+DApKSkUFhbi5eWFyWTi6NGjzJkz\n54b7vPLKK8yfP5/x48ezd+9eEhISeji1iIhjVF5q6ihi5wqrySu+ROs1R8f8vV0ZPyS449pjUaE+\nhIf59ZpC6Wh9pqAF+Xswf1IU63blsmF3Hg/PjHN0JBERp/TVrz7Ahg3rb71hP/L444+TmprKE088\ngc1m46WXXgLAarUSGBjYsZ3VauWtt97i5Zdf5rHHHmPFihWYzWYMBgOvvPKKg9KLiHQfW2sb+SW1\nV689VlhNRU1Tx3qT0UBkiHfHkbFBEX4E+LrplKQv0WcKGsDCyVHsPFLEpr35zBgVRpC/h6MjiYhI\nH+Dl5cUbb7xx3fLf/e53ne4HBwfz8ssvAzBkyBA+/PDD6/YREXFm1XXN7WXs8tGx3OJLtNjaOtb7\neLowJiGI+Ag/4sN9iQnzxc1FI9vuRJ8qaO6uZh6bPYg/rDvB3zKy+e5DIxwdSUSk13j66W/w85+v\nZMCAARQXX+T/+X+eJzg4hIaGBhobG/n3f3+BYcP0fVNERNq1trVRaK3rNFzRWtXYsd5ggIHBV46O\n+TIowo9gfw8dHbtHfaqgAUwaHsrWrAvsP1XK6fxKhkRZbr2TiEg/MHPmHL744nMeffRr7NixnZkz\n5xAfn8DMmbM5cGAf77//J1599f9zdEwREXGQ2oYWzhddue5YDeeLamhqae1Y7+VuZmRcIIMifImP\n8CM2zBcPtz5XJxyuz72jRoOBpXMH88qf9/NB+ll+8s0JfeKCdSLSt6SdW8fB0qNd+phjQkbyyKDF\nN10/c+YcfvObX/Poo19j587tfO97/86HH/6FDz74Cy0tLbi7u3dpHhER6b3a7HYultWRXVRz+bpj\n1Vws7zyTbHiQF/Hhvh3njw0I9MSoo2Pdrs8VNIC4cF+mjhjArmPF7DhSxKz7IhwdSUTE4eLi4ikv\nt1JSUsylS5fYsSODoKAQ/uM/fsapUyf4zW9+7eiIIiLSTRqabJwvujrNfXZRDQ1Nto71bq4mhsVY\niA/3Y9BAP+LCffFyd3Fg4v6rTxY0gEdnxXPgtJW0z88zITEUT/c++1JFxAk9Mmjxlx7t6i5Tpkzn\nnXd+y4wZs6iqqiQ+vn3q9+3bt2Gz2W6xt4iIOAO73U6RtZa9Ry92nDtWaK3Dfs02oRaPjsk8BkX4\nERHkpVFnvUSfbS0WHzcWT43mH9vPs3ZXDo8n6fozIiKzZs3hO995mvfe+4DGxgZeeWUF27al8+ij\nXyM9/VPWr//E0RFFROQu1Da0cCK3gmM5FRzPqaDy0tWp7l3NRgZH+neUsbgIX3w9XR2YVr5Mny1o\nAPdPiGT7oSLS919g5uhwwgK9HB1JRMShhg4dzvbtezruv//+Rx23p0+fBcCiRV/By8uL+npdMFRE\npLeytbaRXVjN8dz2QpZ78VLHETIvdzPTRocTHezFoIF+DAz2xmwyOjSv3L4+XdBczCYeTxrEf60+\nxqqt5/g/j412dCQRERERkTtmt9sprWzoOEJ2Mr+Spub2GRZNRgMJkf4Mjw1gRGwA0aE+hIb6YrXq\ngzZn1KcLGsDYwcEkRvlzJLuco+fLGRkX6OhIIiIiIiK3VNfYwsncyo6jZGXVV69BFhrgyYiYAIbH\nBjAkyl/T3fchff5v0nB52v2X/riXD7ecZWi0RYd4RURERKTXaW1rI6foEsdyyjmeW8H5ohrsl8ct\nerqZGTckuP0oWUwAQf4ejg0r3abPFzSAyBBvZt0XQcbBQrZlFTJvQqSjI4mIiIiIYK26ZthiXgUN\nTe3DFo0GA/ERfh1HyWLCfDAZdZChP+gXBQ3goRmx7D1RwpqdOUweHoqPZq4RERERkR7W0GTjVF4l\nxy4PWyytbOhYF+TnzqRhAxgeE8DQaH88dR2yfqnfFDRfT1cenB7LB1vO8vGOHJalDHF0JBERERHp\n49ra7OQU13D88lGy7MIa2i6PW3R3NTEmIYgRse1HyUIsng5OK71BvyloAHPGRpBxqJCMQ4XMHhNB\nZIi3oyOJiIiISB9TXt3I8cvXJDuZW0Fdow0AgwFiw3wZfnnYYly4r+ZGkOv0q4JmNhlZkpzAf/7t\nMB+kn+GFpWMwGHTFdBERERG5e43NNk7nV3WcS1ZcUd+xLtDXjXFDQhgRG0BitAVvDw1blC/Xrwoa\nwMi4QEbFB3Iku5ysM1bGDQlxdCQRERERcSJtdjv5JZc6hi2evVBNa1v7sEU3FxOj4wMZfnnY4oAA\nTx0QkDvS7woawJLkBI7nVLBq6zlGxQfiYjY5OpKIiIiI9GKVl5o4nFPJ7qNFHM+poLahBQADEDXA\nhxGXLxIdH+GnYYtyT/plQRsQ4Mnc8QPZvLeAT/cVsGhKjKMjiYiIiEgv0tTSypmCqo6jZIVldR3r\nLD5uTB8ZxvDYAIbGWPDV7ODShfplQQN4YGosu44Vs25XHlNHhGHxcXN0JBERERFxELvdTkFpbfvk\nHucrOHuhCltr+7BFV7OREXEBTBoRTkyIF+GBGrYo3affFjRPdzOPzIzjT5tOk7Y9m2cWD3N0JBER\nERHpQdW1TRy/fD2y47mV1NQ1d6yLCvHuOI8sYaAfLmYTwcE+WK2XHJhY+oN+W9AAZowKZ1tWIV8c\nK2bO2IHEhfs6OpKIiIiIdJMWWytnLlR3DFssKK3tWOfr5cqU4QMYERvAsBgLft4aXSWO0a8LmtFo\nYOncBH7x14P8Nf0MqcvGYdThahEREZE+wW63U1hW11HIThdU0WJrA9ovvzQsxsKI2PYZFwcGe2nY\novQK/bqgAQyJsjA+MYT9p0rZc7yEKSMGODqSiIiIiNylmvpmTlwZtphTQVXt1WGLEcFeDI9pn20x\nIdIfNxfN5C29T78vaABfmx3P4XNl/D3jHGMGB+HuqrdFRERExBnYWts4d6G64yLReSVXzxHz9nBh\n8rBQhscGMCwmQJPCiVNQEwGC/D2YPzGKtbty2bA7n0dmxjk6koiI9CJ1dXUsX76c6upqWlpaeO65\n53jnnXeor6/H09MTgOXLlzNixIiOfVpaWnjxxRcpKirCZDLx2muvERkZ6aiXINJn2O12iivqOwrZ\n6fwqmlpaATAZDSRG+TM8NoARsYFEhnrr9BVxOipoly2cHM3OoxfZtCefmaPCCPL3cHQkERHpJVav\nXk1sbCzPP/88JSUlPPXUUwQHB/Paa68xePDgG+6zbt06fH19WblyJTt37mTlypX8+te/7uHkIn1D\nbUMLJ/MqOXa+nOO5FVTUNHWsCwv0bB+2GBfA4Eh/jYQSp6d/wZe5uZr46ux4/rD2BH/bdo7vPjzS\n0ZFERKSXsFgsnD59GoCamhosFsst98nMzOShhx4CYOrUqaSmpnZrRpG+xNbaxvmimo6jZLkXa7Bf\nXuflbmZCYkj7FPgxAQT6uTs0q0hXU0G7xuRhoWzNusD+01ZO51cyJOrWP4BFRKTvW7RoEWlpacyb\nN4+amhp+//vfs3LlSt58800qKyuJj48nNTUVd/ervyiWlZUREBAAgNFoxGAw0NzcjKur602fx2Lx\nxGy+90kLgoN97vkxeoozZQXnyutMWYOCvLlYXsfB01YOni7lyLkyGppsQPuwxWFxgYwZHMyYISHE\nD/THZHTcsEVnel+dKSs4V97uzHpbBe3nP/85hw8fxmAwkJqayqhRowAoKSnhhz/8Ycd2BQUFPP/8\n8zzwwAPdk7abGQwGvj53MD/7037+mn6WFd+cgNGB3wBERKR3WLNmDeHh4bz77rucOnWK1NRUnn32\nWYYMGUJUVBQrVqzg/fff55lnnrnpY9jt9puuu6Kysv6eszrThXSdKSs4V15nyGq32zmeW8GJ/Gr2\nnyimrLqxY12oxYMpw9sn90iMsuDhdvVX1ory2hs9XI9whvf1CmfKCs6VtyuyflnBu2VB27t3L3l5\neaxatYrs7GxSU1NZtWoVAKGhofzlL38BwGazsWzZMpKSku4prKPFhvkybcQAvjhWzOdHiph9X4Sj\nI4mIiINlZWUxffp0ABITEyktLSUpKQmTqf1oV1JSEhs2bOi0T0hICFarlcTERFpaWrDb7V969Eyk\nv2hrs5N1xsq6zFzyS9rLloebmXGDg9uHLcYGEKy5AKQfu2VBy8zMZO7cuQDEx8dTXV1NbW0t3t7e\nnbZbvXo1KSkpeHl5dU/SHvTo7Hj2n7Gy+vPzTEwMwdPdxdGRRETEgaKjozl8+DApKSkUFhbi6enJ\nM888w5tvvomvry979uwhISGh0z7Tpk1j06ZNzJgxg23btjFp0iQHpRfpHWytbew+XsKG3XkUV9Rj\nACYODeGRpMEEepkxGY2OjijSK9yyoJWVlTF8+PCO+wEBAVit1usK2t///nf+53/+p+sTOoC/txuL\np0Tzj+3n+eSLXJYkJ9x6JxER6bMef/xxUlNTeeKJJ7DZbPz0pz+lsrKSb37zm3h4eBAaGsr3v/99\nAJ599lnefvttFi5cyK5du1i6dCmurq68/vrrDn4VIo7R3NLKjiMX2bQnj/KaJkxGA9NHhbFwcjQD\nAjydamibSE+440lCbjSG/uDBg8TFxV1X2m7EWU6A/vqCYew8WsyWAxd4OCmBgSH39nw66bF7KGv3\ncaa8yto9nClrd/Py8uKNN964bvnChQuvW/b2228DdFz7TKS/amiykXGwkM37Cqipa8bFbCR53EDm\nT4zSzIsiX+KWBS0kJISysrKO+6WlpQQHB3faJiMjgylTptzWEzrTCdBfnRXPf60+ytsfHeb/PDb6\nrh/HmT4ZUtbu4UxZwbnyKmv36KqsKnki/U9tQwvp+wtI33+B+iYb7q4mFk6O5v4Jkfh66TxMkVu5\nZUGbNm0ab731FkuWLOH48eOEhIRcd6Ts6NGjN/wU0dmNHRzE0GgLR7LLOZJdzqj4QEdHEhEREemV\nKi818em+fDIOFtHU0oq3hwsPz4gledxAnc8vcgduWdDGjh3L8OHDWbJkCQaDgRUrVpCWloaPjw/z\n5s0DwGq1EhjY98qLwWBgaXICK/64lw+3nGVYjAWzSSewioiIiFxhrWpg4558dh4pwtZqx9/blYdn\nxjFrdDhurvd+WotIf3Nb56Bde60zaJ9i+Fpr167tukS9zMAQb2bfF8G2g4VszSrk/gmRjo4kIiIi\n4nCFZXVsyMxjz4kS2ux2gv3dWTg5mqkjwnAx6wNtkbt1x5OE9EcPzYhlz4kS1uzMYfLwUHw9NX5a\nRERE+qecizWsz8wj64wVgIggLxZNiWbC0BBNlS/SBVTQboOPpysPzojlg/SzfLwjhydThjg6koiI\niEiPsdvtnCmoYl1mHsdzKgCIDfNh8ZQYRicEYTQYHJxQpO9QQbtNc8ZEkHGwkO2HCpl9XzhRoZqZ\nTERERPo2u93O0fMVrMvM5dyFagASo/xZNDWGYdEWDCpmIl1OBe02mU1GliYn8Ku/HebDLWd5YekY\nfVMSERGRPqmtzc6BM1bW78olv7QWgNHxgSyaGsOgCD8HpxPp21TQ7sCIuEBGxwdyOLucrDNWxg0J\ncXQkERERkS5ja21j9/ESNuzOo7iiHgMwcWgICydHa/SQSA9RQbtDjycncCynglVbzzEqPhAXs6aP\nFREREefW3NLKjiMX2bQnj/KaJkxGA9NHhbFwcjQDAjwdHU+kX1FBu0MDAjyZO34gm/cWsHlvAYun\nxjg6koiIiMhdaWiykXGwkM37Cqipa8bFbCR53EDmT4wi0M/d0fFE+iUVtLvwwNRYdh0rZn1mHtNG\nhmHxcXN0JBEREZHbVtvQQvr+AtL3X6C+yYa7q4mFk6O5f0Ikvl66nJCII6mg3QVPdzOPzornvY2n\n+Mf2bL69eJijI4mIiIjcUuWlJj7dl0/GwSKaWlrx9nDh4RmxJI8biKe7i6PjiQgqaHdt+sgwth64\nwK5jxcwZG0F8uGY0EhERkd7ZjAP5AAAgAElEQVTJWtXAxj357DxShK3Vjr+3Kw/PjGPW6HDcXHU+\nvUhvooJ2l4xGA0vnJvCLvx7kg/SzpC4bp4s0ioiISK9SWFbHhsxc9pwopc1uJ9jfnYWTo5k6IgwX\ns9HR8UTkBlTQ7sGQKAsTEkPYd6qUPcdLmDJigKMjiYiIiHC2oJL/3XCSrDNWACKCvFg0JZoJQ0Mw\nGVXMRHozFbR79NiceA6dK+PvGecYMzgId1e9pSIiItLz7HY7ZwqqWJeZx/GcCgBiw3xYPCWG0QlB\nGukj4iTUJu5RkJ8H8ydGsXZXLht25/HIzHhHRxIREZF+xG63c/R8Besyczl3oRqAUYOCmDd+IMOi\nLRhUzEScigpaF1g4OZqdRy+yaU8BM0aFE+zv4ehIIiIi0se1tdk5cMbK+l255JfWAjA6PpBFU2OY\nct9ArNZLDk4oIndDBa0LuLmaeGx2PO+sPcHft53juw+PdHQkERER6aNsrW3sPl7Cht15FFfUYwAm\nDg1h4eRookJ9HB1PRO6RCloXmTQslK1Zhew/beVUXiWJ0RZHRxIREZE+pLmllR1HLrJpTx7lNU2Y\njAamjwpj4eRoBgR4OjqeiHQRFbQuYjC0T7v/sz/t54MtZ1nxzQkYjRrzLSIiIvemoclGxsFCNu8r\noKauGRezkeRxA5k/MYpAP3dHxxORLqaC1oViw3yZNnIAXxwt5vMjRcy+L8LRkURERMRJ1Ta0kL6/\ngPT9F6hvsuHuamLh5GjunxCJr5ero+PdM1ubjbKGCqqMZVRW1wFgv/a/drBfvt1+/8pt+623vXLb\nfvV+p7X2G2x7zfPccK3djm+TB9XVDTfNde22V57+xtteXX7tPp3XXn0N122LnasPdXU/e/vGAPhU\nudPSYMfD7I67yb39q9kNd7M7HiZ3zEazJpDppVTQutijs+LZf9pK2vbzTEwMwdPdxdGRRETkHtXV\n1bF8+XKqq6tpaWnhueeeIzg4mJdffhmj0Yivry8rV67Ew+PqJFFpaWm88cYbREVFATB16lSeffZZ\nR70EcSKVl5r4dF8+GQeLaGppxdvDhYdnxJI8bqDT/V7RZm+jorEKa30ZJQ1WSuvLsNaXUVpvpbyx\n8rpyJD3HZDBdLm9ul8vb5T8mdzyuKXLul4td56J3eZnJHZPR5OiX0ueooHUxf283Fk+J5h/bz/PJ\nF7ksSU5wdCQREblHq1evJjY2lueff56SkhKeeuopgoKCePHFFxk1ahS/+MUvSEtL4xvf+Ean/RYu\nXMjy5csdlFqcjbWqgY178tl5pAhbqx1/b1cenhnHrNHhuLn23l+C7XY71c01l4tXGaUNl7/WWylr\nKMdmb71uHx8Xb2L9ognxDCLEz0JDfUvHumuP6hgwYLi64vKyf/rvNcuvbm24ZpcbrDdcu+2tn/fK\nPW9vN2prm9rvG26wLQauPsytnvfGr63To/1Tphu95qvP23lbbx83rJXVNNoaaGhtotHWSKOtkQZb\nI42X71+5XdpQRlNrM3fDxejScXTOw+TR6ShdR7G75r672aO9AF5zVM/N5HZXz91XqaB1g/snRPL5\n4SK2HLjArPvCCQ7WjEoiIs7MYrFw+vRpAGpqarBYLPzud7/D29sbgICAAKqqqhwZUZxYYVkdGzJz\n2XOilDa7nWB/dxZOjmbqiDBczEZHx+tQ11JPab31mhJmvXxkrIzmG/xy725yJ9w7rL2EeQYT4hF0\n+XYQHuarR5uDg32c5pIATpfV4/azttnbaLQ10dh6ubh1un21zF253+n25fWVjVW0tNnuKq+H2R03\nk9s/lbmbFb0bH9VzNbr0iWGbKmjdwMVs4vGkBH6TdpRVW88xKnGAoyOJiMg9WLRoEWlpacybN4+a\nmhp+//vfd5Sz+vp61qxZwxtvvHHdfnv37uWZZ57BZrOxfPlyhg0b1tPRpRfLuVjD+sw8ss5YAYgI\n8mLRlGgmDA3BZHRMMWu0NWG9XL5K68spbbB2HBmrs9Vft72L0YVgj8D2AuYZdLmEtd/2dvHqE78s\n9xdGgxFPFw88Xe7ter62Nts15a6JRltDpzJ3/RG89q82WrjUWM+l5kuU2qy02dvu6jV0GrL5T8M3\nO5W7y7evDOe8tui5GB1bkVTQusmYhCCGRls4kl3O/pMlRAdp+lsREWe1Zs0awsPDeffddzl16hSp\nqamkpaVRX1/Ps88+y9NPP018fHynfUaPHk1AQACzZ8/m4MGDLF++nLVr137p81gsnpjN9z6UzZlG\nbjhTVrj3vHa7nePny/lb+hkOXi5mCZH+fG3uYCYOG9ClM0DfLGtLawsltWVcrC3l4qUSii6VUnyp\nlIuXSqlsrL5ue5PBSIh3EEN84gnzDiHM5+qfAA9/jIZ7L5PO9O9AWbuf3W6npbWFelsj9S0NNLR0\n/nqjZQ22RuqbGzr2qWyqoqGl8a7Oc3QxmvFwccfTxeOarx54mttvT2UcicGDuuGVt1NB6yYGg4Gl\nyQms+ONe/nvNMVZ8czxmU+8ZpiAiIrcvKyuL6dOnA5CYmEhpaSnNzc1897vfZfHixTzyyCPX7RMf\nH99R2saMGUNFRQWtra2YTDcvYJWV1x+huFNONwTLSbLCveW12+0cPV/Ousw8zl1oL0GJUf4smhrD\nsGgLBoOB8vLaLssaEOjJ6YKCq0MRrzkvrKKx6rpfWg0YsLj7k2hJ6BiSeOXIWKC75YYTQdjroLyu\n7p6zOtO/A2XtPjfOa8AFT1zwxNcIuF3+c5va7G00tzZ/+RE8WyMNre1DOq+739xIRUP1dUN4q5pq\nCCT0nl/vzaigdaOBId7MHhPBtqxCth64wP0ToxwdSURE7kJ0dDSHDx8mJSWFwsJCvLy8ePfdd5k4\ncSKPPfbYDff5wx/+QFhYGIsXL+bMmTMEBAR8aTmTvqmtzc6BM1bW78olv7S9gI2OD2TR1BgGRfjd\n02Pb7XaqmqqxNpRRcmV2xMszJZY1VtDadv3kHL6uPsT5xRB6pYRdHpYY7BGIi8m5ZogUuRWjwdgx\nvNHf7e7/f2tta6Wptal9yGZrI8OiYqmuaOzCpJ2poHWzh2fEse9kKWu+yGXyiAH4ejr/dUtERPqb\nxx9/nNTUVJ544glsNhsvvfQSL7zwAgMHDiQzMxOASZMm8b3vfY9nn32Wt99+mwceeIAXXniBDz/8\nEJvNxquvvurgVyE9ydbaxu7jJWzYnUdxRT0GYOLQEBZOjiYq9PaHndntdmpb6jqXsHorpQ3tt5vb\nWq7bx8PsQax/JBaXAEI8r54fFuwRhIdZF7YWuVMmowlPoyeeLu2nLLmaXAAVNKfl7eHC11MSeefj\no3z8+XmenJ/o6EgiInKHvLy8rpsEZOfOnTfc9u233wZgwIAB/OUvf+n2bNK7NLe0suPIRTbtyaO8\npgmT0cD0UWEsnBzNgICbn4/eYGvsVL6una6+wdZw3fauRpf2o1//NDtiiEcwXi6ehIT4OtXwNhG5\nSgWtByyYGsPaHdlsP1zE7DERd/TJmYiIiPR+DU02Mg4WsnlfATV1zbiYjSSPG8j8iVEE+rUftWpu\nbaGsobzjvLDSjuuGWbnUfP35ZyaDiSCPQAb5x3YuYZ7B+Ln6aoZEkT5KBa0HmE1GliYn8Ku/HebD\nLWd5YekYfVMVERHpA2obWkjfX0D6/gvUN9lwdzUwe5I/iYNdqG0r4bPi41jPl1FSb6WqqfqGk3ME\nuFsYGjC44whYsGcQoZ5BWNz8bzg5h4j0bSpoPWREXCD3DQri0LkyDpy2Mj4xxNGRRERE5C602dvI\nK7ey+chJDhfk0+ZyCXNsA4F+TTRyiT32Nvac7ryPn6tv+5Gwy+eChXgGE+oZRKBHoMOvuSQivYu+\nI/Sgx5MGcfR8OX/bdo5R8YG4uuhTMRERkd6utL6MPRf3U3Gmgtyyi1gbyrAb2mdINEbClYvoGMye\nRHsMvDxFfVCn6erdzXcwN7iI9GsqaD0oNMCTeeMj2bQ3n837CnhgaoyjI4mIiMhNlNSVsjlvG/tK\nDtJmbwPA3mrC3uiFa6sPCSERjI2OIcy7fViit4uXgxOLSF+ggtbDFk+NYdexi2zIzGP6yDAsPvpE\nTUREpDe5WFfCptwtHCg5jB07AS5BFJ8Mp/VSAOF+ASyeEsOEoSGYjMZbP5iIyB1SQethnu5mHpkV\nz3sbT/FRRjb/8sAwR0cSERERoLD2Ihtzt3Co9Ch27ER4hzE/OpmPPqnDXtnIi8vGMyjMG6Mm+hKR\nbqSC5gDTR4axNesCmceLSRobQXzE3V/ZXERERO5NwaVCNuZu4bD1GABRPhEsiJnLyKBh7DxykeLy\nMmaODmfa6HBdW0xEup0KmgMYjQa+Pncwr7+fxQdbzpK6bJw+jRMREelheTUFbMxN52jZSQBifKNY\nEJPM8MBEDAYDLbZWPt6Zg4vZyFemxTg2rIj0GypoDjI40p+JQ0PYe7KU3ceLmToizNGRRERE+oXz\n1XlszE3nRHn7XPhxfjEsjJlLYkBCp+uUbs0qpPJSE/MnRRHg6+6ouCLSz6igOdBjswdx8GwZf8/I\nZuzgYNxd9dchIiLSXc5V5bAxJ51TlWcBSPCPY2HsXBL84zsVM4D6RhvrM/PwcDOzcHK0I+KKSD+l\nRuBAgX7uLJgUxSdf5LI+M49HZ8U7OpKIiEifYrfbOVuVzYacdM5WnQdgiGUQC2KSSbDc/Ofupr35\n1Da08OisOLw9XHoqroiICpqjLZgUzY4jF9m8t4CZo8MJ9vdwdCQRERGnZ7fbOV15jg056WRX5wAw\nLGAIC2KTifOL+dJ9q+ua+XRfPn5erswdF9kDaUVErlJBczA3VxOPzYnnnU9O8Ldt53ju4ZGOjiQi\nIuK07HY7JypOszEnnZyafABGBA5lQWwyMb5Rt/UY677IpbmljcfnxODmaurGtCIi11NB6wUmDQ1l\n64FCDpy2cjKvkqHRFkdHEhERcSp2u51j5SfZkJNO/qULAIwOGs78mGSifAfe9uOUVjWQcaiQEH8P\nZowO7664IiI3pYLWCxgMBpbOTeBnf9rPB+lneelbEzAaNe2+iIjIrbTZ2zhSdoJNOekU1BYBMCZ4\nJPNjkhnoc+cFa82O87S22XloZixmk7Gr44qI3NJtFbSf//znHD58GIPBQGpqKqNGjepYd/HiRX7w\ngx/Q0tLCsGHDePnll7stbF8WG+bL9JFh7Dx6kc8PFzF7TISjI4mIiPRabfY2DlmPsTEnnaK6YgwY\nGBcymvkxyYR7D7irxyworWX38RIiQ7yZODS0ixOLiNyeWxa0vXv3kpeXx6pVq8jOziY1NZVVq1Z1\nrH/99dd5+umnmTdvHj/96U8pKioiPFxDAu7Go7Pi2He6lLTPzzNhaAhe7po1SkRE5Fpt9jaySg6z\nMW8rxXUlGDAwIXQs82OSGOAVck+PnbY9Gzvw6Kx4jAaNZBERx7hlQcvMzGTu3LkAxMfHU11dTW1t\nLd7e3rS1tXHgwAF+9atfAbBixYruTdvH+Xm78cDUGD7KyOaTnbksnZvg6EgiIiK9QmtbK/tLDrEp\nbwul9WUYDUYmDxhPSswcQjyD7/nxzxRUcTi7nMGR/oyMC+iCxCIid+eWBa2srIzhw4d33A8ICMBq\nteLt7U1FRQVeXl689tprHD9+nPHjx/P88893a+C+bt74SD4/VMTWrAvMHhNOWKCXoyOJiIg4TGtb\nK3uLs9iUt5WyhnKMBiNTwyaSEjOHII/ALnkOu93OR9uzAfjq7OsvWi0i0pPueJIQu93e6XZJSQlP\nPvkkERER/Ou//isZGRnMnj37pvtbLJ6Yzfc+ZW1wsM89P0ZPupO8//LwSF79417SduTw0r9M6cZU\nN+ZM762ydh9nyqus3cOZskrfY2uzsefiATbnbaW8sRKzwcSMiCnMi5pNoEfXznZ8OLuccxeqGZMQ\nxKAIvy59bBGRO3XLghYSEkJZWVnH/dLSUoKD24cSWCwWwsPDiYpqv67IlClTOHv27JcWtMrK+nuM\n3P5Lg9V66Z4fp6fcad64EC+GRls4cKqULbtzGBUf1I3pOnOm91ZZu48z5VXW7tFVWVXy5E61tLaQ\neXEfn+ZlUNlUhdloZtbAacyLmoXF3b/Ln6+tzc4/tmdjAB6ZGdfljy8icqduOX/stGnT2Lx5MwDH\njx8nJCQEb29vAMxmM5GRkeTm5nasj42N7b60/cSVafcNBvhgyzlsrW2OjiQiItKtmltb2FawkxWZ\nv2DVmY+pbakjKXIGL095ka8NfrBbyhnAnhMlFFrrmDpiABHB3t3yHCIid+KWR9DGjh3L8OHDWbJk\nCQaDgRUrVpCWloaPjw/z5s0jNTWVF198EbvdzuDBg0lKSuqJ3H3ewGBv5oyJYGtWIVsPXOD+iVGO\njiQi0m/V1dWxfPlyqquraWlp4bnnniM4OJiXXnoJgCFDhvDTn/600z4tLS28+OKLFBUVYTKZeO21\n14iMjHRA+t6tqbWZnYW7+Sw/g0vNtbgaXZgbNYvkqJn4unbvEVhbaxurd5zHbDLw4HR9wCwivcNt\nnYP2wx/+sNP9xMTEjtvR0dF88MEHXZtKAHhoRhx7TpSw5otcJg8fgK+Xq6MjiYj0S6tXryY2Npbn\nn3+ekpISnnrqKYKDgzuuDfr888+zfft2Zs2a1bHPunXr8PX1ZeXKlezcuZOVK1fy61//2oGvondp\ntDWx5mQma05+Sm1LHW4mV+6PnkNS5Ax8XHvmSNb2Q0WUVTcyd/xAgvw9euQ5RURu5ZZDHHubvJoC\nvpn2A/504kNK6kodHadbeXu48OD0WBqabKzecd7RcURE+i2LxUJVVRUANTU1+Pv7U1hYyKhRowCY\nM2cOmZmZnfbJzMxk3rx5AEydOpWsrKyeDd1LNdga2ZS7lZ9kvsb7R1Zja2tlQUwyP5uayoPxC3qs\nnDU221j7RQ5uriYWT4npkecUEbkddzyLo6P5uvoQ5BnA3uIs9hUfZGzIKObHJBPuPcDR0brF7DER\nZBwq4vNDRcwZE0FUqE64FxHpaYsWLSItLY158+ZRU1PD22+/zcsvv9yxPjAwEKvV2mmfsrIyAgLa\nr6dlNBoxGAw0Nzfj6to/R0PUtzSQcWEnWwt20mBrwMPswddGLGaCZQKeLj1/9OqzfQXU1LfwlWkx\nGqEiIr2K0xU0i7s//29KKltO7GZj7hYOlB7mQOlh7gseyfyYZCJ9wh0dsUuZTUaWJiewctUhPkg/\ny//99TG6PouISA9bs2YN4eHhvPvuu5w6dYrnnnsOH5+rH5hdewmam7mdbfripWhqm+pYf2YrG85u\npaGlER9XL5aOfJCUhFkOKWYA1bVNbN5XgK+XK99YOAxPd5fb3rc3vbe3oqzdQ1m7jzPl7c6sTlfQ\nAIwGI/eFjGR08AiOlZ9kY84WDlmPcsh6lJFBw1gQk0y0b985EXt4bAD3DQri0LkyDpy2Mj4xxNGR\nRET6laysLKZPnw60n4fd1NSEzWbrWF9SUkJISOfvzSEhIVitVhITE2lpacFut9/y6FlfuhRNbXMd\nWwo+Z/uFL2hqbcbbxYuH4hcyI2IK7mY36qpseAbjkKyrtp6lvtHG0uQE6i41Unep8bb26y3v7e1Q\n1u6hrN3HmfJ2RdYvK3hOWdCuMBgMjAwaxojAoZysOMPG3HSOlp3gaNkJhgUOYUHMXOL8oh0ds0s8\nnjSIo+fLWbX1HKPiA3F1ufdPWEVE5PZER0dz+PBhUlJSKCwsxMvLi4iICPbv38/48eP59NNPWbZs\nWad9pk2bxqZNm5gxYwbbtm1j0qRJDkrfs2qaL5Gev50dhbtpbm3G19WHxbH3My1iMm4mxw8lrKhp\nZMuBQgJ93Zg9JsLRcUREruPUBe0Kg8HAsMAhDA0YzJnKbDbmpnOi/DQnyk+TaElgfkwyCRbnvvhk\naIAn8yZEsmlPPpv3FfDA1BhHRxIR6Tcef/xxUlNTeeKJJ7DZbLz00ksEBwfzk5/8hLa2NkaPHs3U\nqVMBePbZZ3n77bdZuHAhu3btYunSpbi6uvL66687+FV0r6qmatLzt7OzcA8tbS34ufryYNwCpoZP\nxNV0+0MIu9vHO3Owtbbx0Iw4XMxON1eaiPQDfaKgXWEwGBgSMIghAYM4V5XDxpx0TlWe5VTlWRL8\n45gfk8wQyyCnPYfrgakx7Dp6kfWZuUwfGYbFx83RkURE+gUvLy/eeOON65b/9a9/vW7Z22+/DdBx\n7bO+rrKxis/yM/iiaC+2NhsWN3/uj57DlLDxuPSiYgZQVFbHF0cvEh7kxZThfXNyMRFxfn2qoF1r\nkH8s3x/zL+RU57ExdwvHy09x9tB5Yn2jWRA7l2EBg52uqHm4mXlkVjzvbTzFRxnn+JcHhjs6koiI\n9FPlDZV8mr+N3UX7sNlbCXS3kBKdxKSwcZiNvfPXi9Wfn8duh0dnxmE0OtfvACLSf/TO76BdKNYv\nmu+Ofpq8mgI25W7lSNlxfnv4XaJ8BrIwdi4jAoc6VVGbPjKMbVmFZB4vIWnsQOIj/BwdSURE+pGy\nhnI2525jd/F+2uxtBHkEMj86iYkDxmIy9t7zo88X1XDgjJX4cF/uSwhydBwRkZvq8wXtimjfSP6v\nUU9x4VIRm3K3cMh6jN8deY+B3uHMj0lmdPBwjIbePxbdaDSwdG4Cr7+fxV/Tz/KjJ8dhdKKCKSIi\nzqm03srm3G3sLcmizd5GiGcQ86OTGR96X68uZtB+iYOPMs4B8NXZ8U71wayI9D/9pqBdMdAnnG+P\nXEZRbTGb87ZyoOQw/33sL4R5hTI/JpmxIaN6fVEbHOnPxKEh7D1ZSuaxYqaNDHN0JBER6aOK60rZ\nlLuV/SUHsWNngFcoC6KTGBs6utf/vLziRG4lp/KrGBEXwJAoi6PjiIh8qX5X0K4I9x7At4Z/nYUx\nc9mct419JQf54/G/siHnM1Kik3r9J4KPzR7EobNlfLQ9m3FDgnF37bd/lSIi0g2KaovZlLuFrNIj\n2LET7jWABbFzuS94hNMUM4A2u52PtmcD8OjMeAenERG5tX7/W32oVwhPDnucBTFz+TRvK7uLD/Dn\nk6vYkJvefrJzLx1TH+jnzvxJUXzyRS7rM/N4dJZ+6IiIyL27cKmIjblbOGQ9CkCkdzgLYucyMmiY\nUxWzKw6ctpJXfIlJw0KJHnDzC8OKiPQW/b6gXRHsGcg3hj7G/Ji5fJafQWbRXt4/9Xc25qZzf/Qc\nJoeNx6WXzUq1YHI0O45cZPPeAmaMDifE38PRkURExEnl11xgY+4WjpQdByDaJ5IFsclON5nWtWyt\nbaRtz8ZkNPDQjFhHxxERuS29q3H0AoEeFpYMeZiU6Dmk52/ni6I9fHg6jU25W5gXNbtXXXDTzcXE\nY3PieeeTE/x96zmee2SkoyOJiIiTya3JZ2NOOsfKTwEQ6xvFgth5Tnk5mn+28+hFSiobmDMmglCL\np6PjiIjcFhW0m7C4+/PY4Ae5PzqJLfnb2VGYyd/PrmFz3lbmRs1iesRk3Eyujo7JpKGhbM0q5MAZ\nKyfzKhkarZOfRUTk1s5X57IhJ52TFWcAiPeLYWHsPIZYBjl9MQNoamnlk505uJqNPDAtxtFxRERu\nmwraLfi5+fBIwmLmRc9ma8EOtl/4grRz6/g0bxvJUTOZGTEFd7O7w/IZDAa+PjeBn723nw/Sz7Li\nW+MxGZ3vHAEREekZZyvPszE3ndOV7dPOD/aPZ0HsXBL84/pEMbti64ELVNU2s2hKNP7ebo6OIyJy\n21TQbpOPqzcPxi9gbtQsthXsIOPCF6zJ3kh63nbmRM5gduRUPMyOOQcsZoAv00aFsfPIRT4/fJE5\nYyIckkNERHonu93OmcpsNuamc7bqPACJlgQWxM5lkH/fOzerrrGF9Zl5eLmbWTApytFxRETuiAra\nHfJy8WRxXApJkTPZfuELthbsYF3OZrYUbGf2wOnMiZyOl0vPj3N/dGYc+0+Vsvrz80wcGoKXe+84\nT05ERBzHbrdzquIsG3LTOV+dC8DwwEQWxCQT6xft2HDdaNOefOqbbDw2Ox5P/TwUESejgnaXPF08\nWBA7lzmR0/m8MJMt+Z+zMTedbQU7mDlwKsmRM/F29eqxPH7ebjwwNYa/Z2SzZmcOX587uMeeW0RE\nehe73c7x8lNszN1Cbk0+ACODhrEgJplo30gHp+teVbVNfLavAH9vV5LGDXR0HBGRO6aCdo/cze7c\nHz2HWQOnsbNwN5/lZ/Bp3jYyCnYyY+AUkiNnEUzPXHdl7vhIth8qYltWIbPviyA8qOcKooiIOJ7d\nbmdf4WFWHV5L/qVCAO4LHsH8mGQiffrH8Pe1X+TSbGtjyfRY3Fx633VMRURuRQWti7iZXEmOmsmM\niCnsKtrLZ/kZbMn/nM8v7GJe/AymhUzF382vWzO4mI08njyIt/5xlA+3nOXfvza6T53wLSIiX+79\nUx+ReXEfBgyMDRnF/JhkIrzDHB2rx5RU1vP54SJCAzyZMar/vG4R6VtU0LqYq8mF2ZHTmBYxid0X\n97E5dxsbzm7j03OfMyV8IvdHzybAvfumwr9vUBDDYiwcy6ngSHY5owcFddtziYhI7xLhHUZS7FSm\nhkwhzCvU0XF63Mc7cmhts/PIzDjNaCwiTksFrZu4GM3MiJjClLAJnKw7wUdHN7CjMJNdRXuZNGAc\nKTFzCPII7PLnNRgMLElO4KX/2ceHW88xPDYAs0k/pERE+oM5kdMJDvbBar3k6Cg9Lq/4EntOlBA9\nwIdxQ4IdHUdE5K6poHUzs9FMUtw0hnkNZ3/JITblbWHXxb3sLt7PhNAxpMT8/+3deXxU9aH//9ds\n2SbrJDNJyJ6wBILseyQsgiJuoGL1am1v7Sbtr/f22n61aEvbW7X2Yn+21mrVb3uttW6AiqLSosgi\nYRMEwk42SCDJZCUbZJvvHwkDESFQMpmZ8H4+HjzIOXPm8OaIOfPO+ZzPmUlsSO+eSBLtocwYncBH\n20v46LMSrpugKYZFRIzx0UcAACAASURBVKR/W7YuH4Dbp2Vg1PB+EfFjKmh9xGQ0MTF+LOPjRrO9\nYhcfFH3E5rLP2FK2nbGxI5mTek2vDke5ZWoam/aWseLTQiZnxRFuDei1fYuIiPiS/cU15BVUMzQl\nimGpnruNQESkL2jsWx8zGoyMix3FwxN+yH3D72FAaBzbyj/n0c2/5cW8v1HacLxX/pzQYAvzpqbT\nfKqd5esKemWfIiIivsblcrFsbefVs9umZWhyLBHxe7qC5iVGg5ExjhGMtl/F7sq9fFC0mh0Vu9hR\nsYuRMVnMSbuG5LDLe37L9NED+GRHKet3HmPmmASSY/tmun8REZG+8vmhSvKPnWDsYDvpA8K9HUdE\n5LLpCpqXGQwGRtiz+D/jfsDCkd8gLTyZnZV7eGLr73l2558prDvyL+/bZDRy56xBuIC/rz6Ey+Xq\nveAiIiJe1tHhYtm6AgwGmJ+T7u04IiK9QlfQfITBYCArOpNhtiEcqDnM+4WryavaT17VfjKjBnF9\n2iwGRqZd8n6zUm2MGhjD54cr2XbAyfhMhwfSi4iI9L3cPWUcq2zk6hHxDIixejuOiEivUEHzMQaD\ngUzbIDJtgzhUk88HRR+xv+YQ+2sOMSgynblpsxgUeWlj7L9yzUB2F1TxxseHGZkRTYDF5MG/gYiI\niOe1tnXw9voCzCYj866+9B9gioj4KhU0HzYoKoNBURkU1BXxQeFH7K0+wO92PE96RCpzU2eRaRt0\nUUUtNiqEa8cn8cHmI6zacoSbsnUiExG5FG+++SYrVqxwL+/cuZORI0e6lysqKpg/fz7f/e533eue\nfvpp3n33XWJjO2fovfnmm1mwYEHfhe7nPtlRStWJU1w3IQlbeJC344iI9BoVND+QHpHK90bdR9GJ\nI3xY9BG7K/fxh50vkhKexPWp1zA8emiPRe3GKal8mlfGyk3FZF8Vr5OZiMglWLBggbtcbdmyhQ8+\n+IDFixe7X//mN7/JLbfccs777r33Xu65554+y3mlaD7VxrsbiwgONHHD5FRvxxER6VWaJMSPpIYn\n890R/85D4/+DUfbhFJ84ynO7/pcntv6Oz515dLg6zvve4EAzt+Wk09LawdKu6YhFROTSPfPMMyxc\nuNC9vHHjRlJTU4mPj/diqivLqi1HaGhuZc6EZEKDLd6OIyLSq1TQ/FBSWALfuupeFk34IWMdIylp\nOM4Lu//K41ue4rPynectatkj4kmJC2PTnnIOl9b1cWoREf+3a9cu4uPjsdvt7nV//etfuffee790\n+w8//JB///d/5zvf+Q5Hjx7tq5j92onGFlZtPUp4iIXZ45O8HUdEpNdpiKMfSwiN5xvD72Zu42w+\nLPqYbeU7+POeV4gtdDAndSZjHSMxGc9MCGI0GPi3WYN4/G/beXX1QR6+dxxGPdBTROSiLV26lPnz\n57uXy8vLaWpqIjk5+Zxtp02bxqRJkxg/fjwrV67kV7/6FX/6058uuP+oqBDM5sufyMlu95/nXl5q\n1rc/3c2plna+fsMwkhKiPJTq/PrzsfUmZfUMf8oK/pXXk1lV0PqBOKuDr2fdydy0WfyjeA2byz7j\npb2v8X7hP7ku9RomxI52F7VBiZFMGOpgy74KcvPKyL5KQ3JERC7W5s2beeSRR9zLa9euZdKkSV+6\n7YgRI9xfz5w5kyVLlvS4/5qapsvOaLeH4XTWX/Z++sKlZq2sbeb9jYXERAQxdmB0n/89+/Ox9SZl\n9Qx/ygr+lbc3sl6o4GmIYz/iCInhnqEL+Pmk/8PVAyZSfbKWv+17g19s+g0bSjfR1tEGwILpAwkw\nG1n6ST7Np9q8nFpExD+Ul5djtVoJCAhwr9u9ezeZmZlfuv2vfvUrtm3bBnROLDJo0KA+ydmfvbOh\nkLZ2F/OnpmM26SOMiPRP+u7WD0UH27gr8zZ+MflBpiVOoa6lnlcPLOfnub9hbclGwkNNXD8phbrG\nFt7fVOztuCIifsHpdGKz2c5ZFx0d3W35Zz/7GdA58+OSJUu45557ePHFF3n44Yf7NG9/U+JsYGNe\nGYl2KxOHxXo7joiIx2iIYz8WFRTJHYPncV3KTFYfWcv60k28cfBtVhV9xPTEHKIiTKzacoSpIwfg\niAz2dlwREZ82fPhwXnzxxW7rnnvuuW7LdrudX/7ylwAMGTKE1157rc/y9XdvrSvABdw2LQOjUfdP\ni0j/pStoV4CIwHBuG3QT/z3lJ8xOnk5z+yneKVgJQ9eAvYDXPt7n7YgiIiLndbi0jh2HKhmUGMGI\njOie3yAi4sdU0K4gYQGhzBs4l/+e/BPmpMwEQzuW5APsty7j5R3v0dx20tsRRUREunG5XCz9pPP5\nnbdNy8Cg2YdFpJ/TEMcrUGiAlZsy5nBNcg5v7f+YT49vZFPNOnZu3EJ28jgiTTbiQhzEWu1EBUbq\nZCgiIl6zu6Cag0drGZkRzeCkSG/HERHxOBW0K1iIJYS7r7qRpqNJbD2+hYDko6wu2NBtmwBTALEh\n9s7C1lXa4kIc2ENisBj1z0dERDynw+Vi2dp8DMCt0zK8HUdEpE/oE7ZwR04mO56vpa1uED9fmEVx\nVSlljRWUN1VQ3uTkeGM5R+tLu73HgIGYYBtx1q7iFuIgzuogLsROiCXES38TERHpT7bsK+doRQOT\ns2JJcoR6O46ISJ+4qIL22GOPsXPnTgwGA4sWLTrn4ZtxcXGYTJ0PQl6yZAmxsZr+1p9EhAZyU3Yq\nb67JZ+VH1dwxLYtxsaPcr3e4Oqg+WUNZYwVlTRWUNzopb+r8enflPnbTfZKRMEtoV3GzE2eNJTbE\nTmyIg6igCIwG3fYoIiI9a2vv4K11BZiMBm6Zmu7tOCIifabHgrZlyxaKi4t5/fXXyc/PZ9GiRbz+\n+uvdtnnhhRewWq0eCymeN2tsEht3l7FuRymf7StnzsRkrhmbSFCAGaPBSExwNDHB0QxnaLf3NbQ0\ndpW2zsJ2usAdri3kUG1Bt20DjJbOsmZ1dN3j1lniHMExWEyWvvzrivRbLpeLDlcHrR2ttHa00dbR\ndtbvrbR1tHf93vaF17tvb6+OZHzUOP1QRbxm/c5jOGtPcs2YRD0KRkSuKD0WtNzcXGbNmgVARkYG\ndXV1NDQ0EBqqoQb9icVs5OF7x7Jpv5NlHx9i2doC/rH1KDdMSmH66AQCLKYvfV9ogJWBAWkMjEzr\ntr6lvZWKptNX2pzdCtzRhmPdtjVgIDrYRtzZ5a1ryKRVwyXFj7hcLtpd7ectRVUE4Kw+4X79dFE6\ns9xZlE5vf3ofF97+TLk6vezC1St/n4zJA4kJtvW8oUgvO9XSzopPiwi0mLgxO9XbcURE+lSPBa2y\nspKsrCz3ss1mw+l0ditoixcvprS0lLFjx/LAAw9ccNa/qKgQzOYv/7B/Kez2sMveR1/yl7xJCVHM\nnZLGO+vyeXttPq99fJh/bCvhjlmDuXZiChbzxf80PQEbMKTbug5XB5WN1ZTWl1F6opzSE2Ucqy+j\n5EQZeVX7yava32378MBQEsLjSQiLZUB4HInhcQwIjyMmJArwn+MK/pUV/Cuv3R7WWY462mnpaKWt\nvY2WjlZa29tobe8qL+2ttJz1dWt7Z+Fpae8sPS1nrTv9Pvc+3O9pPWt/p/+M7vvsrXJ0IWajGYvJ\njMVoxmKyEGIOxmwyE2C0dK43WdyvWUwWAozms163nPW6mQCTBUu391kIMJmJCo4kPszh8b+LyJdZ\n/dlR6hpbuGlKKhHWAG/HERHpU5c8SYjL1f3Dxw9+8AOmTp1KREQE3/ve91i1ahVz5sw57/trapou\nPeUX2O1hOJ31l72fvuJPee32MJoaTjJ7TAKThzr4YHMxH20r4bnlu3hz9UFuzk5lylVxmIz/+rAn\nA4EkmlNItKXAWT+cb2hppLzJSVlT+Zn73Bor2O88zD7noW77sBgtJITHEh0Q7b7qFmd1YA+OIcAH\nh0v6078B8G7e9o52GtuaaGhppKG181dja2O35YaWrnWtTbR0nKKlvXevHF2I2WDCbOwsOGajmQBj\nACGWECyBls7i1LW+83cLFuOZ7cNDQ2g92dH1ugWz0dT1+5e978uXzUZTnww7tIf1zr8Bfyr64hsa\nmlt5f9MRQoMtXDch2dtxRET6XI8FzeFwUFlZ6V6uqKjAbre7l+fNm+f+Oicnh4MHD16woIn/CA22\nsGD6QK4dl8TKTcV8sqOUv3ywn/c3FXPL1WlMGBqL0dh7z0gLDbASGmAlIzK12/rW9lYqmiu7zSxZ\n1ljBsfpyitpLum1rwEB0UNRZ97nZ3cMlQy26T7KvuVwuTraforG1kfquUlX/JYXr7OWmtuaL2neg\nKYBQi5WYEBt0GN1lxmIyYzacKTWnl91Fp9ty51Ujs+FMUXLvp2v7s5dNl1mO/K2oi3jD+5uKaT7V\nxldmDiQkSJNNi8iVp8fvfNnZ2Tz99NPceeed7NmzB4fD4R7eWF9fz3/+53/y7LPPEhAQwNatW7nu\nuus8Hlr6VkRoIP82azBzJiTz3sYi1u86zvPv7mVlbjHzpqYxZrDdow+ztpgsJITGkxAa3219dIyV\ng0dLOicmOWuikvJGJ3uq9rPnC8MlQy3WrpklTz8aoHOWSVtQpCZCuEhtHW1dher0Fa4GGlqbul3V\nOruANbY20uZq73G/RoMRqyWEyMAIEkLjCQ0IJdRiJdQSQqgllFBLCNYAq/vrUIvVPbGMSo9I/1F9\n4iQffVZCVFggM8ckeDuOiIhX9FjQxowZQ1ZWFnfeeScGg4HFixezfPlywsLCmD17Njk5OXzlK18h\nMDCQYcOG6epZP2YLD+LeOZnMmZTCu58WsjGvjGfeyiMlNoz5OWlclR7t0aL2RUaDkejgKKKDo8iK\n7n6vW2NrU9cQye5DJgvqismvK+q2rcVoxuF+GPeZiUocIXafHC7ZW1wuF81tzV1XsZpoaDldthro\nKG3FWVd7TgE72X7yovYdZAoiNMBKYlBCV9GyYg0IIcwSivV08QoIdRewYHNQn/7bERHftOLTIlrb\nOph3dRqWXrhfXUTEH13U2IEf/ehH3ZYzMzPdX3/ta1/ja1/7Wu+mEp/miAzmvhuGMXdSCu9sKGTL\nvgqeenMXGQnh3Do1naGp3p/1zWoJIT0ilfSI1G7rW9tbcTZXdXs0QHnXLJOlDce7bWvAgC0oilhr\nZ3k7+9EAYQG+N4tpa3vrmXu0WhtpbDnrapa7gJ09rLCJDldHj/s1GUyEWqxEB0edKVuWzuGop5dD\nz1q2WkIwGzUsSUQuzfGqRjbsOk58dAhTrorzdhwREa/Rpyj5l8VHW/nuLcO5YXIDb68vYMehSv7n\ntc/JTI5kfk46gxIjvR3xHBaThQGhcQwI7X7y73B1UHuqjvJG51nPc+v8fW/VAfZWHei2vdUS0nlv\n2xceDRAdHNUrwyU7XB00tTV3v0/r7Ekyzpkso5FT7S0Xte8QczChFiv24Oiuq1ndC1aoxUqiw05b\nowGrxUqQKVBXt0TE495aX0iHy8WtOemXNRGViIi/U0GTy5bkCOX/u20EhcdP8Nb6AvIKqnn8b9u5\nKj2a+TlppMaFeztij4wGI7agKGxBUQyNHtzttabWJvez3M6eZbKwrpiCLwyXNBvNOIJjzswsGWIn\n1hpLcEQiVc013e/ROu/MhJ1Xty5mRkKz0UyoxYojOObcq1oB1nMKmNUcgsnY87Ahe3QYzg7d1yUi\nfaPw+Am27a8gLT6MMYPtPb9BRKQfU0GTXpMWH85/3TGKg0dreWtdAbsLqthdUMWYwXbmXZ1GosP3\nhgVejBBLCOkRKaRHpHRb39rRhrOpsnOIZNcjAU4/mPtYY1n3nWzt+c8xYCDEEkyoJZTYEHu3khX2\nJcMKrRYrgaYAXd0SEb+3fG0+ALdPy9D3NBG54qmgSa8bnBTJ//m30ewrruGtdQVsP+hkx0EnE4bF\ncsvVacTZQrwdsVdYjOYvHS7pcrmoPVXnnlGyrKmCRlcDpnYLoQEh3WYiPHuijBBLsGaTFJErzt6i\navYU1ZCVZvOJe5hFRLxNBU08wmAwMCzVxtCUKHblV/HW+gI27y1ny75ysofHc3N2KjGRwd6O6REG\ng4GooEiigiIZauscLqmp4EVEzuVyuVjWdfXstmnpXk4jIuIbVNDEowwGAyMHxnBVRjQ7Djp5a30h\nG3YfJ3dPGTkjB3DjlFSiwgK9HVNERLxg4+7jFB6vZ3ymwy/uVxYR6QsqaNInjAYDY4c4GD3IzpZ9\n5by9oZA1O0pZv+s4M0YnMHdyChHWAG/HFBGRPtLe0cHL7+/DaDAwP0dXz0RETlNBkz5lNBqYlBXH\n+KEONu4uY8Wnhfxz21HW7ixl1tgk5kxMJjS4/z4cWkREOn26u4xSZwPTRg3oN/cmi4j0BhU08QqT\n0cjUkQOYlBXH+l3HeHdjEe9vKmbNjhKuHZ/MteOTCA7UP08Rkf6opbWddzYUEmA2cnN2mrfjiIj4\nFH0CFq+ymI3MHJPI1VfFs2ZHKStzi3lnQyGrtx3l+kkpXDMmkcCAnp/bJSLiSW+++SYrVqxwL+fl\n5TF8+HCampoICem8+vPggw8yfPhw9zatra089NBDHDt2DJPJxOOPP05SUlKfZ/dFH28vpab+FLfN\nGKj7kEVEvkAFTXxCgMXEdROSmTZqAKu3lfDh5iMs/SSff2w5wg2TU5k+egAWs4qaiHjHggULWLBg\nAQBbtmzhgw8+4PDhwzz++OMMHjz4S9/z3nvvER4ezpNPPsmGDRt48skneeqpp/oytk9qOtnGytwi\nggPN3DZzECcbT3k7koiIT9FDl8SnBAWYuXFKKr+5fzI3TUnlVFsHr350iIf+tIlPdpTS1t7h7Ygi\ncoV75plnWLhwYY/b5ebmMnv2bACmTJnC9u3bPR3NL3y45QiNJ9uYOymZsBBNDiUi8kW6giY+KSTI\nwvycdGaNS+SDzUf4+LMS/rrqAO9vKuaWq9OYlBXr7YgicgXatWsX8fHx2O12AH7/+99TU1NDRkYG\nixYtIigoyL1tZWUlNlvng5eNRiMGg4GWlhYCAq7cUlLX2MI/th4hIjSAWeM03FNE5MuooIlPCwsJ\n4I4ZA7lufBIrc4v55PNS/u/KfazMLearc4cyJCEco8Hg7ZgicoVYunQp8+fPB+Dee+9lyJAhJCcn\ns3jxYl555RXuu+++877X5XL1uP+oqBDMvTCc224Pu+x9eMLy9btoae3gmzdnkjggEvDdrOfjT3mV\n1TOU1XP8Ka8ns6qgiV+ICA3k32YPZs7EZN7dWMSGXcf5n799RqLdyryp6YweFINBRU1EPGzz5s08\n8sgjAO7hiwAzZ87k/fff77atw+HA6XSSmZlJa2srLperx6tnNTVNl53Rbg/D6ay/7P30toraZj7I\nLcIRGcyodBtOZ73PZj0ff8qrrJ6hrJ7jT3l7I+uFCp7uQRO/YgsP4mtzMnn0WxOZOS6J0spG/rB8\nN//90jZ2F1Rd1E+oRUT+FeXl5VitVgICAnC5XHz961/nxIkTQGdxGzRoULfts7Oz+fDDDwFYs2YN\nEydO7PPMvuTt9QW0d7iYn5OO2aSPHyIi56PvkOKXHFEh/PCuMfz3fRMZn+mgqKye//+NnTz+ynb2\nF9d4O56I9ENOp9N9T5nBYOCOO+7g61//OnfffTdlZWXcfffdANx///0AzJ07l46ODu666y5eeeUV\nHnjgAa9l97Yj5fVs3lNOsiOU8UMd3o4jIuLTNMRR/NqAGCv3zxvODeX1vL2+kM8PV/KbV3cwNCWK\nW3PSyUiI8HZEEeknhg8fzosvvuhenjt3LnPnzj1nu2effRbA/ewzgeXrCnABt03P0H3DIiI9UEGT\nfiE5Nowf3D6CgmMneHt9AXmF1Tz68meMyIhm/tR0UuL856ZTEZH+5ODRWnblVzEkKZLhaTZvxxER\n8XkqaNKvpA8I57++MoqDR2tZvq6AXflV7MqvYuwQO/OuTiPBHurtiCIiVwyXy8XStflA59UzTeYk\nItIzFTTplwYnRfLgv41mb1ENy9cV8NkBJ9sPOJmYFcst2WnE2kK8HVFEpN/bmV/F4ZI6Rg+KYaCG\nnIuIXBQVNOm3DAYDWWk2hqVGsTO/irfWFbBpTzlb9laQfVUcN2WnEhMR7O2YIiL9UkeHi2Vr8zEY\n4NacdG/HERHxGypo0u8ZDAZGDYxhREY02w84eWt9Aet3HWdjXhnTRg3ghsmpRIUFejumiEi/snlv\nOaXORrKvitPwchGRS6CCJlcMo8HAuEwHYwbb2by3nHc2FPLx9lLW7zrOjNEJzJ2UQrj1wg+RFRGR\nnrW2dfDW+gLMJgO3XJ3m7TgiIn5FBU2uOEajgcnD4xg/1MHGvDJWfFrIP7YeZe3nx5g1LpE5E5Ox\nBlm8HVNExG+t/byUyrqTzB6XpKHkIiKXSAVNrlhmk5GckQOYnBXHup3HeG9jEStzi/l4eynXTUhi\n9rgkggP1v4iIyKVoPtXGuxuLCAwwccOUFG/HERHxO/r0KVc8i9nINWMTuXpEPGu2l/L+pmLeXl/I\n6m0lXD8pmZljEgm0mLwdU0TEL/xz21Hqm1q55eo0wkM0bFxE5FIZvR1AxFcEWkzMmZjME9+dzPyp\nabR3uHhzTT4PPpfL6m1HaW3r8HZEERGfVt/UwoebjxAabOHa8UnejiMi4pdU0ES+IDjQzE3Zafzm\n/sncOCWVU63t/H31IX7yfC5rPy+lrV1FTUTky6zMLeZkSzs3TUnVEHERkX+RCprIeViDLNyak84T\n353MnAnJ1De18tKHB3j4hU1szDtOR4fL2xFFRHxGVd1JPt5eSnR4ENNHJ3g7joiI31JBE+lBeEgA\nd8wcyBPfncw1YxKpPnGKF9/bx0//72a27q+gw6WiJiLyzqeFtLV3MG9qGhazPl6IiPyrNP5A5CJF\nhgZy97WDuW5iEu9tLGLDrjKefTuPJEco86emM3JgNAaDwdsxRUT63LHKRj7dfZyEGCuTs+K8HUdE\nxK+poIlcopiIYL5+/VCun5TCig2FbNpTzu+X7SItPpz5OWlkpdpU1ETkirJ8XQEuF9w6LR2jUd//\nREQuhwqayL8oNiqEb92UxdzJqbyzvoBtB5z89vWdDE6MYH5OOkOSo7wdUUTE4/KP1bH9oJOMhHBG\nDYzxdhwREb+ngiZymRJirCycfxXFZfW8vb6AnflVPPH3HWSlRjEvJ52MARHejigi4hEul4tln+QD\ncPu0DI0eEBHpBSpoIr0kJS6M/1gwkvzSOt5aX8Ceohr2FH3GqIEx/PvNwwkL0E3zItK/7CmqZv+R\nWq5Kj9aoARGRXqKCJtLLMhIi+NGdozlwpIbl6wr4/HAl//HbT0iJC2NCpoNxmQ7skcHejikiclk6\nXC6WfVIAwG3T0r2cRkSk/1BBE/GQIclRPHT3GPYUVfPJ58f5/KCT4rJ63vwkn9S4MMYPdTB+iIMY\nlTUR8UPb9ldQXF7PxGGxJMeGeTuOiEi/oYIm4kEGg4HhadHMmJBK4ZFqth90snV/BfuKaigqq+fN\nNfmkxYcxPjOWcZl2YiJU1kTE97W1d/DWugJMRgPzpqZ5O46ISL+igibSR0KDLeSMHEDOyAHUN7Ww\n41AlW/eVs6+4lsLj9byx5jBp8eGMz3SorImIT9uw+zjlNc3MGJNAbFSIt+OIiPQrKmgiXhAWEtCt\nrLmvrBXXUHj8BG+sOUz6gK6yNsRBdESQtyOLiABwqrWdFRsKCbAYuWlKqrfjiIj0OypoIl4WFhLA\ntFEJTBuVwInTZW1fBfuP1FBw7ASvf3yYjNNlLdOBLVxlTaSvvfnmm6xYscK9nJeXx6uvvsovf/lL\njEYj4eHhPPnkkwQHn7nyvXz5cn73u9+RnJwMwJQpU7j//vv7PHtv++izEmobWrhhcgqRoYHejiMi\n0u+ooIn4kPCQAKaPSmD6qARONJ65srb/SA35x07w2seHyUgI77xnbYhdZU2kjyxYsIAFCxYAsGXL\nFj744AN+9atf8dBDDzFixAieeOIJli9fzt13393tfXPnzuXBBx/0RmSPaDzZyvu5xViDzFw/Mdnb\ncURE+iUVNBEfFW4NYProBKaPTqDudFnbV86Bo7Xkl57gtY8OMTAhwn1lLSpMP8kW6QvPPPMMS5Ys\nITg4mNDQUABsNhu1tbVeTuZ5H2w6QtOpNhbMyCAkyOLtOCIi/dJFFbTHHnuMnTt3YjAYWLRoESNG\njDhnmyeffJLPP/+cl19+uddDilzpIqwBzBidwIzTZe1ABVv3V3DgSC2HS+t49aNDDEqMYFzXPWsq\nayKesWvXLuLj47Hb7e51TU1NvPPOO/zud787Z/stW7Zw33330dbWxoMPPsiwYcP6Mm6vqqk/xept\nR4kMDeCaMYnejiMi0m/1WNC2bNlCcXExr7/+Ovn5+SxatIjXX3+92zaHDx9m69atWCz6aZqIp0VY\nA5gxJpEZYxKpazjFZ133rB08WsuhkjpeW32mrI1VWRPpVUuXLmX+/Pnu5aamJu6//36+8Y1vkJGR\n0W3bkSNHYrPZmD59Ojt27ODBBx/k3XffveD+o6JCMJtNl53Tbu/955K9ubaAlrYOvn39UBIGRPba\nfj2R1ZP8Ka+yeoayeo4/5fVk1h4LWm5uLrNmzQIgIyODuro6Ghoa3MM6AH7961/zwx/+kD/84Q8e\nCyoi54oIDWTmmERmjkmktuEUnx3ovGft0NFaDpbU8erqQwxKimR8poOxQ+y6oV/kMm3evJlHHnkE\ngLa2NhYuXMiNN97Irbfees62GRkZ7tI2evRoqquraW9vx2Q6fwGrqWm67Ix2exhOZ/1l7+ds5dVN\nrNpUTKwthJFpUb22f09k9SR/yqusnqGsnuNPeXsj64UKXo8FrbKykqysLPeyzWbD6XS6C9ry5cuZ\nMGECCQkJlxVSRC5PZGgg14xN5JqxidTUn3Lfs3boaC0Hj9by938eZHBSZNcwSDsRKmsil6S8vByr\n1UpAQAAAL7zwyh5L6QAAGTNJREFUAhMmTHBPHvJFL7zwAvHx8dx4440cPHgQm812wXLmy95aX0CH\ny8WtOemYjEZvxxER6dcueZIQl8vl/rq2tpbly5fzl7/8hfLy8ot6vy8P3/Akf8qrrJ7Rl1nt9jAG\np8dw55yhVNU1s3HXcTbsLGVvYTUHjtby99UHGZ4eQ/bIAUwZEU9U2LmzQerYeoay+i+n04nNZnMv\nv/LKKyQmJpKbmwvAxIkT+f73v8/999/Ps88+y0033cSPf/xjXnvtNdra2nj00Ue9Ff2yFJfVs2Vf\nBSlxYYwbYu/5DSIicll6LGgOh4PKykr3ckVFhfvm6E2bNlFdXc3dd99NS0sLR44c4bHHHmPRokXn\n3Z+vDt/wJH/Kq6ye4e2skzLtTMq0U1N/im1dE4zszq9kd34lf3prF0O6hkGOGeIgwhrg9byXQlk9\no7ey9qeSN3z4cF588UX38oYNG750u2effRaAuLi4fjFx1rJ1+QDcPj0Dg8Hg5TQiIv1fjwUtOzub\np59+mjvvvJM9e/bgcDjcwxvnzJnDnDlzACgpKeEnP/nJBcuZiHhXVFggs8clMXtcEtUnTrrvWdt/\npJb9R2r52z8PkpkcxYxxSQweEE64NcDbkUXEi/YX15BXUM3QlCiyUm09v0FERC5bjwVtzJgxZGVl\nceedd2IwGFi8eDHLly8nLCyM2bNn90VGEfEAW3gQs8cnMXt8Z1nbdsDJ1v3l7CuuYV9xDQYDZCZH\ndV1ZsxMeorImciVxuVwsXXvm6pmIiPSNi7oH7Uc/+lG35czMzHO2SUxM7BdDOUSuRLbwIK4dn8S1\nXWVtX8kJPtl2xF3W/vaPg2SmdA2DHGwnTGVNpN/bcaiSgmMnGDvETlp8uLfjiIhcMS55khAR6d9s\n4UHMm2Yne5iDqrqT7nvW9hbVsLeohpdXHWRoSiTjh8YyZrCd0GA9/1Ckv+nocLFsbT4GA9yak+7t\nOCIiVxQVNBE5r+iIIK6bkMx1E5KprGtm2/7Oe9b2FNWwp6iGv354gKGpUe4rayprIv3Dxrwyjlc1\nMXVEPPHRVm/HERG5oqigichFiYkIZs7EZOZMTKayttl9z9qewmr2FFbz8qoDDE3pLGujVdZE/FZr\nWzvvbCjAbDJyy9Vp3o4jInLFUUETkUsWE3mmrDlrmzuHQe6rIK+wmrzCav66qvuVNWuQypqIv1iz\n4xhVJ05x3YQkbOHnPiNRREQ8SwVNRC6LPTKY6yemcP3EFCpqm9m2v/OetbyCavIKqvnrhwcYlmrr\nurIWo7Im4sOaT7Xx3sYiggNN3DA51dtxRESuSCpoItJrHJHBzJ2UwtxJKVTUNLF1fwXb9jvZXVDF\n7oIqTB8ayErrKmuDYghRWRPxKau2HKGhuZX5Oekapiwi4iUqaCLiEY6oEG6YnMoNk1Mpr2lyX1nb\nlV/FrvwqTEaVNRFfcqKxhVVbjxJuDWD2uERvxxERuWKpoImIx8WeXdaqT19Z617WhqfZGD/UwaiB\ndkKC9K1JpK+9t7GIUy3t3D4tg6AA/T8oIuIt+g4sIn0q1hbCjVNSuXFKKmVnlbWd+VXszK/CbNrP\n8LRoxmc6GDUohuBAfZsS8bTK2mbW7CglJiKIaaMGeDuOiMgVTZ98RMRr4mwh3DQllZumpHK8qrFr\nGKSTzw9X8vnhSswm41lX1lTWRDzl7Q2FtHe4mJ+Tjtlk9HYcEZErmj7tiIhPiI+2clN2Gjdlp3G8\nqtF9Ze3ssnZVeuc9ayNV1kR6TYmzgdy8MhLtoUwcFuvtOCIiVzx9whERnxMfbeXm7DRuzk7jWGXX\nlbUDFew4VMmOQ2eVtaEOrpmo5zSJXI7lawtwAbdNS8doMHg7jojIFU8FTUR82oAYKzdfncbNV6dR\n2lXWtu0/U9ZefG8fGQPCGZ5mIystmtS4MIxGfcgUuRiHS+r4/HAlgxIjGJER7e04IiKCCpqI+JGE\nGCsJV6dxy9VplDob2HbAyf4jtRw8WsOhkjreWl+INcjMsFQbWWk2hqfZsIXrCpvIl3G5XCz95DAA\nt0/PwKCrZyIiPkEFTUT8UoI9lAR7KN+0h1F4pJp9xTXsKawir7CarV3PXAOIjw5xl7UhSVEEBpi8\nnFzEN+wuqOZgSR2jBsYwKDHS23FERKSLCpqI+L3QYAvjMx2Mz3Tgcrkoq24ir7CaPYXV7D9Sw+pt\nJazeVoLZZGBQYiRZaTayUm0kxYbqnhu5InW4XCxbm48BuDUn3dtxRER8wieffMT06df0uN2jjz7K\njTfexoABCR7JoYImIv2KwWAgPtpKfLSV2eOSaG3r4HBpHXu6Ctu+4hr2FdewlHzCQywM6yprWWk2\nIkMDvR1fpE9s2VvO0YoGJmfFkegI9XYcERGvO378GKtXr7qogvbwww/jdNZ7LIsKmoj0axazkaEp\nUQxNieL26RmcaGxhb1FnWcsrqmbTnnI27SkHINEe2jXZiI1BiREEWDQcUvqftvYO3lpfgMloYN7U\nNG/HERHxCb/97RPs27eHqVPHc+2113P8+DGeeuqPPP74L3E6K2hubuYb3/g22dlT+epXv8r3v/9f\nrFnzEY2NDRw5UkxpaQk/+MEDTJ6cfdlZVNBE5IoSbg1gUlYck7LicLlclDobO4dDFlVz8GgtJc4G\nPtxyBIvZyJCkruGQaTYSYqyaREH6hXU7j+GsPck1YxOxRwZ7O46IyDne+Piw+17y3jI+08EdMwee\n9/W77voqy5e/QVpaBkeOFPHHP75ITU01EyZM4vrrb6S0tISf/vQhsrOndntfRUU5S5b8nk2bNvLO\nO8tU0ERELofBYCDREUqiI5Q5E5NpaW3nYEmtezhkXtcvgMjQAHdZG5ZqIzwkwMvpRS7dqZZ2Vnxa\nRKDFxI1TUr0dR0TEJw0dmgVAWFg4+/btYcWK5RgMRk6cqDtn2xEjRgHgcDhoaGjolT9fBU1EpEuA\nxcTwtGiGp3U+D6qm/pR7OOSeomo+3V3Gp7vLMADJcWGdwyFTbQxMjMBsMno3vMhF+Oe2o5xobOGm\nKalEWPVDBhHxTXfMHHjBq12eZrFYAPjnPz/kxIkTPPPMi5w4cYJvfvOr52xrMp25HcLlcvXKn6+C\nJiJyHlFhgWRfFU/2VfF0uFwcLW8gr7CKPYXVHCqpo7isnpW5xQRaTGQmnxkOGWcL0XDIfubNN99k\nxYoV7uW8vDxeffVVfv7znwMwZMgQfvGLX3R7T2trKw899BDHjh3DZDLx+OOPk5SU1Jexu2lobuWD\nzcWEBluYMzHZazlERHyR0Wikvb2927ra2lri4wdgNBpZu/ZjWltb+ySLCpqIyEUwGgykxIWREhfG\nDZNTOdnSxoEjte6razvzq9iZXwVAdHiQ+9lrQ1OjsAZZvJxeLteCBQtYsGABAFu2bOGDDz7g0Ucf\nZdGiRYwYMYIHHniAtWvXMm3aNPd73nvvPcLDw3nyySfZsGEDTz75JE899ZS3/gq8v6mY5lPt3Dkz\njeBAnf5FRM6WkpLGgQP7iY8fQGRk57Mhp0+fyUMP/Rd79+Zxww0343A4+MtfXvB4Fn2HFhH5FwQF\nmBk5MIaRA2MAqKxrZm9RDXmF1ewrqmbdzmOs23kMgwHS4sMZnmYje3QithAzJqOGQ/qzZ555hscf\nf5x77rmHESNGADBjxgxyc3O7FbTc3FzmzZsHwJQpU1i0aJFX8gJUnzjJR5+VYAsPZMYYzzy3R0TE\nn0VFRbF8+cpu6+LjB/DSS6+5l6+99noA7PYwnM560tPPDMNMTx/IH/7wfK9kUUETEekFMRHB5IwM\nJmfkADo6XBSWnXBPNpJfeoKCYydY8WkRwYEmhqbY3MMhHZpFz6/s2rWL+Ph4TCYT4eHh7vXR0dE4\nnc5u21ZWVmKz2YDOoTMGg4GWlhYCAvr+3q8VnxbS2tbBLVenYTHr8REiIr5MBU1EpJcZjQYyBkSQ\nMSCCm7PTaDrZxv4jNeSX1fPZ3nK2H3Sy/WDnh3lHZDBZ6TaGp9rITInS0DMft3TpUubPn3/O+ou5\nMfxitomKCsHcCwXKbg9zf320vJ4Nu46TFBvKLdMHYfKxCW3OzuoP/CmvsnqGsnqOP+X1ZFZ9EhAR\n8bCQIDNjBtu5LjsdZ049FTVN7mn89xXXsGZ7KWu2l2I0GMhICO96WHY0qXFhGI2abMSXbN68mUce\neQSDwUBtba17fXl5OQ6Ho9u2DocDp9NJZmYmra2tuFyuHq+e1dQ0XXbG00NvTvvzO7vpcMHNU9Ko\nrm687P33pi9m9XX+lFdZPUNZPcef8vZG1gsVPBU0EZE+5ogKwREVwowxibS1d1Bw7IR7spHDpXUc\nKqnjrfWFWIPMDEu1uSccsYUHeTv6Fa28vByr1eouWenp6Wzbto1x48bxj3/8g69+tfv0y9nZ2Xz4\n4YdMnTqVNWvWMHHixD7PXHj8BNsOOEmLD2fM4Jg+//NFROTSqaCJiHiR2WRkcFIkg5MimZ+TTkNz\nK/uKa9hTWEVeYTVb91ewdX8FAPHRIe6yNiQpisAA3UvUl5xOp/ueMoBFixbxs5/9jI6ODkaOHMmU\nKVMAuP/++3n22WeZO3cuGzdu5K677iIgIIBf//rXfZ552dp8AG6fnqFHP4iI+AkVNBERHxIabGF8\npoPxmQ5cLhdl1U3kdU02sv9IDau3lbB6Wwlmk4FBiV3PXku1kRQbilEfwD1q+PDhvPjii+7lgQMH\n8ve///2c7Z599lkA97PPvGVPUTV7i2rISrMxNCXKazlEROTSqKCJiPgog8FAfLSV+Ggrs8cl0drW\nweHSOvfskPuKa9hXXMNS8gkPsTCsq6xlpdmIDA30dnzxIpfLxbJPuq6eTcvwchoRkf7j9ttv4v33\nV/a84WVQQRMR8RMWs5GhKVEMTYni9ukZnGhsYW9RZ1nLK6pm055yNu0pByDRHto12YiNQYkRBFg0\nHPJK8tkBJ0Vl9UwY6iAlzn9mRRMRERU0ERG/FW4NYFJWHJOy4nC5XJQ6GzuHQxZVc/BoLSXOBj7c\ncgSL2ciQpEj3s9cSYqy6H6kfa2/vYPm6AowGA/Onpns7joiIX/jGN+7msceeJC4ujrKy4/zkJw9g\ntztobm7m5MmT/PCHP2bYsOF9kkUFTUSkHzAYDCQ6Qkl0hDJnYjItre0cLKl1D4fM6/oFEBka4C5r\nw1JthIf0/YOTxXM+2naUsuompo8aQKwtxNtxREQu2fLD77GjYnev7nO04ypuHXjjeV/PyZnBp5+u\n47bb7mD9+rXk5MwgI2MQOTnT+eyzrbzyyks8+uj/9Gqm81FBExHphwIsJoanRTM8LRqAmvpT7uGQ\ne4qq+XR3GZ/uLsMAJMeFdQ6HTLUxMDECs489yFguXktrO39ftR+L2chN2WnejiMi4jdycmbwhz88\nxW233cGGDWv5/vd/yGuvvcyrr75Ma2srQUF996gbFTQRkStAVFgg2VfFk31VPB0uF0fLG8grrGJP\nYTWHSuooLqtnZW4xgRYTmcmR5IxNYmRalGaG9DMfby+lqu4k109KJipME8WIiH+6deCNF7za5Qnp\n6RlUVTkpLy+jvr6e9es/ISbGwU9/+t/s37+XP/zhqT7LooImInKFMRoMpMSFkRIXxg2TUznZ0saB\nI7Xuq2s786vYmV/Fr78zCUeUhsj5k90FVYQGW5g7KcXbUURE/M7kyVfz/PN/ZOrUadTW1pCRMQiA\ntWvX0NbW1mc5VNBERK5wQQFmRg6MYeTAGACq6k6CxUR0iMXLyeRS3XfDUCIiQzB1dHg7ioiI35k2\nbQbf/e43+N//fZWTJ5v51a8Ws2bNam677Q5Wr/4HK1eu6JMcKmgiItJNdEQQdnsYTme9t6PIJbKF\nB2GPtuq/nYjIv2Do0CzWrt3sXn7llaXur6++ehoAN9xwM1arlaYmz32f1Z3gIiIiIiIiPkIFTURE\nRERExEeooImIiIiIiPgIFTQREREREREfoYImIiIiIiLiIy5qFsfHHnuMnTt3YjAYWLRoESNGjHC/\n9sYbb7B06VKMRiOZmZksXrwYgx5sKiIiIiIicsl6vIK2ZcsWiouLef3113n00Ud59NFH3a81Nzez\ncuVKXnnlFV577TUKCgrYsWOHRwOLiIiIiIj0Vz0WtNzcXGbNmgVARkYGdXV1NDQ0ABAcHMxLL72E\nxWKhubmZhoYG7Ha7ZxOLiIiIiIj0Uz0WtMrKSqKiotzLNpsNp9PZbZvnn3+e2bNnM2fOHJKSkno/\npYiIiIiIyBXgou5BO5vL5Tpn3be//W3uvfdevvWtbzF27FjGjh173vdHRYVgNpsu9Y89h90edtn7\n6Ev+lFdZPcOfsoJ/5VVWz/CnrCIiIv1FjwXN4XBQWVnpXq6oqHAPY6ytreXQoUOMHz+eoKAgcnJy\n2L59+wULWm+UMxERkf6ot0qxP5Vrf8oK/pVXWT1DWT3Hn/J6MmuPQxyzs7NZtWoVAHv27MHhcBAa\nGgpAW1sbDz30EI2NjQDs3r2btLQ0j4UVERERERHpzwyuLxuz+AVLlixh27ZtGAwGFi9ezN69ewkL\nC2P27NksX76cV155BbPZzJAhQ/jFL36hafZFRERERET+BRdV0ERERERERMTzehziKCIiIiIiIn1D\nBU1ERERERMRHqKCJiIiIiIj4iEt+Dlpfe+yxx9i5cycGg4FFixYxYsQI92sbN27kt7/9LSaTiZyc\nHL73ve95MemFs86cOZO4uDhMps7HDCxZsoTY2FhvRQXg4MGDLFy4kK9//evcc8893V7ztWN7oay+\ndmx/85vf8Nlnn9HW1sZ3vvMdrr32WvdrvnZcL5TVl45rc3MzDz30EFVVVZw6dYqFCxcyY8YM9+u+\ndFx7yupLx/VsJ0+e5MYbb2ThwoXceuut7vW+dGylO386P4J/nSN1fvQcnSN7n86RnuWV86PLh23e\nvNn17W9/2+VyuVyHDx923XHHHd1ev/76613Hjh1ztbe3u+666y7XoUOHvBHT5XL1nHXGjBmuhoYG\nb0T7Uo2Nja577rnH9cgjj7hefvnlc173pWPbU1ZfOra5ubmub37zmy6Xy+Wqrq52TZs2rdvrvnRc\ne8rqS8d15cqVrueff97lcrlcJSUlrmuvvbbb6750XHvK6kvH9Wy//e1vXbfeeqtr2bJl3db70rGV\nM/zp/Ohy+dc5UudHz9E50jN0jvQsb5wffXqIY25uLrNmzQIgIyODuro6GhoaADh69CgRERHEx8dj\nNBqZNm0aubm5PpnVFwUEBPDCCy/gcDjOec3Xju2Fsvqa8ePH87vf/Q6A8PBwmpubaW9vB3zvuF4o\nq6+ZO3cu3/rWtwA4fvx4t5+m+dpxvVBWX5Wfn8/hw4eZPn16t/W+dmzlDH86P4J/nSN1fvQcnSM9\nQ+dIz/HW+dGnhzhWVlaSlZXlXrbZbDidTkJDQ3E6ndhstm6vHT161BsxgQtnPW3x4sWUlpYyduxY\nHnjgAa8+L85sNmM2f/l/fl87thfKepqvHFuTyURISAgAS5cuJScnx32Z3teO64WynuYrx/W0O++8\nk7KyMp577jn3Ol87rqd9WdbTfO24PvHEE/z0pz/l7bff7rbeV4+t+Nf5EfzrHKnzo+foHOlZOkf2\nPm+dH326oH2Ry48e2fbFrD/4wQ+YOnUqERERfO9732PVqlXMmTPHS+n6F188tqtXr2bp0qX8+c9/\n9mqOi3G+rL54XF977TX27dvHj3/8Y1asWOH1k+GFnC+rrx3Xt99+m1GjRpGUlOS1DHL5/On8CDpH\n9hVfPa46R3qGzpG9y5vnR58e4uhwOKisrHQvV1RUYLfbv/S18vJyr17iv1BWgHnz5hEdHY3ZbCYn\nJ4eDBw96I+ZF8bVj2xNfO7br16/nueee44UXXiAsLMy93heP6/mygm8d17y8PI4fPw7A0KFDaW9v\np7q6GvC943qhrOBbxxXgk08+4aOPPuKOO+7gzTff5I9//CMbN24EfO/Yyhn+dH6E/nOO9MVjeyG+\neFx1jux9Okd6hjfPjz5d0LKzs1m1ahUAe/bsweFwuIdDJCYm0tDQQElJCW1tbaxZs4bs7GyfzFpf\nX899991HS0sLAFu3bmXQoEFey9oTXzu2F+Jrx7a+vp7f/OY3/OlPfyIyMrLba752XC+U1deO67Zt\n29w/vaysrKSpqYmoqCjA947rhbL62nEFeOqpp1i2bBlvvPEGCxYsYOHChUyZMgXwvWMrZ/jT+RH6\nzznSF4/t+fjicdU50jN0jvQMb54fDS4fHxexZMkStm3bhsFgYPHixezdu5ewsDBmz57N1q1bWbJk\nCQDXXnst9913n89mfemll3j77bcJDAxk2LBh/PSnP/Xqpee8vDyeeOIJSktLMZvNxMbGMnPmTBIT\nE33u2PaU1ZeO7euvv87TTz9NWlqae93EiRMZMmSIzx3XnrL60nE9efIkDz/8MMePH+fkyZN8//vf\np7a21ie/F/SU1ZeO6xc9/fTTJCQkAPjksZXu/On8CP5zjtT50XN0jvQMnSM9r6/Pjz5f0ERERERE\nRK4UPj3EUURERERE5EqigiYiIiIiIuIjVNBERERERER8hAqaiIiIiIiIj1BBExERERER8REqaCIi\nIiIiIj5CBU1ERERERMRHqKCJiIiIiIj4iP8HDmSCkU8gqdQAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f709d49ecf8>"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot performance\n",
    "trainer.plot_performance()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 51
    },
    "colab_type": "code",
    "id": "xRphYPaVWTzY",
    "outputId": "c2267e55-ce0c-4e79-a03f-2daf1e8df80d"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test loss: 0.50\n",
      "Test Accuracy: 82.8%\n"
     ]
    }
   ],
   "source": [
    "# Test performance\n",
    "trainer.run_test_loop()\n",
    "print(\"Test loss: {0:.2f}\".format(trainer.train_state['test_loss']))\n",
    "print(\"Test Accuracy: {0:.1f}%\".format(trainer.train_state['test_acc']))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "z_Zrw9rIWTwU"
   },
   "outputs": [],
   "source": [
    "# Save all results\n",
    "trainer.save_train_state()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "bsWE23n-U-Cy"
   },
   "source": [
    "## Using GloVe embeddings"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "HH_328MjWoM-"
   },
   "source": [
    "We just used some randomly initialized embeddings and we were able to receive decent performance. Keep in mind that this may not always be the case and we may overfit on other datasets with this approach. We're now going to use pretrained GloVe embeddings to initialize our embeddings. We will train our model on the supervised task and assess the performance by first freezing these embeddings (so they don't change during training) and then not freezing them and allowing them to be trained. \n",
    "\n",
    "```python\n",
    "pretrained_embeddings = torch.from_numpy(pretrained_embeddings).float()\n",
    "self.embeddings = nn.Embedding(embedding_dim=embedding_dim, \n",
    "                               num_embeddings=num_embeddings, \n",
    "                               padding_idx=padding_idx, \n",
    "                               _weight=pretrained_embeddings)\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "r1EAWPBEKoaS"
   },
   "outputs": [],
   "source": [
    "def load_glove_embeddings(embeddings_file):\n",
    "    word_to_idx = {}\n",
    "    embeddings = []\n",
    "\n",
    "    with open(embeddings_file, \"r\") as fp:\n",
    "        for index, line in enumerate(fp):\n",
    "            line = line.split(\" \")\n",
    "            word = line[0]\n",
    "            word_to_idx[word] = index\n",
    "            embedding_i = np.array([float(val) for val in line[1:]])\n",
    "            embeddings.append(embedding_i)\n",
    "\n",
    "    return word_to_idx, np.stack(embeddings)\n",
    "\n",
    "def make_embeddings_matrix(words):\n",
    "    word_to_idx, glove_embeddings = load_glove_embeddings(embeddings_file)\n",
    "    embedding_dim = glove_embeddings.shape[1]\n",
    "    embeddings = np.zeros((len(words), embedding_dim))\n",
    "    for i, word in enumerate(words):\n",
    "        if word in word_to_idx:\n",
    "            embeddings[i, :] = glove_embeddings[word_to_idx[word]]\n",
    "        else:\n",
    "            embedding_i = torch.zeros(1, embedding_dim)\n",
    "            nn.init.xavier_uniform_(embedding_i)\n",
    "            embeddings[i, :] = embedding_i\n",
    "\n",
    "    return embeddings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "FZBLYjtWKoX5"
   },
   "outputs": [],
   "source": [
    "args.use_glove_embeddings = True"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "id": "Gj77_BmoHA3s",
    "outputId": "d15e120b-4725-49a2-e5a5-cc9f79026f8e"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<Embeddings(words=3407, dim=100)>\n"
     ]
    }
   ],
   "source": [
    "# Initialization\n",
    "dataset = NewsDataset.load_dataset_and_make_vectorizer(df=split_df, \n",
    "                                                       cutoff=args.cutoff)\n",
    "dataset.save_vectorizer(args.vectorizer_file)\n",
    "vectorizer = dataset.vectorizer\n",
    "\n",
    "# Create embeddings\n",
    "embeddings = None\n",
    "if args.use_glove_embeddings:\n",
    "    embeddings_file = 'glove.6B.{0}d.txt'.format(args.embedding_dim)\n",
    "    words = vectorizer.title_vocab.token_to_idx.keys()\n",
    "    embeddings = make_embeddings_matrix(words=words)\n",
    "    print (\"<Embeddings(words={0}, dim={1})>\".format(\n",
    "        np.shape(embeddings)[0], np.shape(embeddings)[1]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 204
    },
    "colab_type": "code",
    "id": "CC4ofLswt-D3",
    "outputId": "9ff430ec-4a60-4572-cc8d-5dc5d0eb13e1"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<bound method Module.named_modules of NewsModel(\n",
      "  (embeddings): Embedding(3407, 100, padding_idx=0)\n",
      "  (conv): ModuleList(\n",
      "    (0): Conv1d(100, 100, kernel_size=(2,), stride=(1,))\n",
      "    (1): Conv1d(100, 100, kernel_size=(3,), stride=(1,))\n",
      "    (2): Conv1d(100, 100, kernel_size=(4,), stride=(1,))\n",
      "  )\n",
      "  (dropout): Dropout(p=0.1)\n",
      "  (fc1): Linear(in_features=300, out_features=100, bias=True)\n",
      "  (fc2): Linear(in_features=100, out_features=4, bias=True)\n",
      ")>\n"
     ]
    }
   ],
   "source": [
    "# Initialize model\n",
    "model = NewsModel(embedding_dim=args.embedding_dim, \n",
    "                  num_embeddings=len(vectorizer.title_vocab), \n",
    "                  num_input_channels=args.embedding_dim, \n",
    "                  num_channels=args.num_filters, hidden_dim=args.hidden_dim, \n",
    "                  num_classes=len(vectorizer.category_vocab), \n",
    "                  dropout_p=args.dropout_p, pretrained_embeddings=embeddings, \n",
    "                  padding_idx=vectorizer.title_vocab.mask_index)\n",
    "print (model.named_modules)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 102
    },
    "colab_type": "code",
    "id": "44gVZbKbHA7R",
    "outputId": "18b4c379-3ff5-476d-a297-6ae023d33279"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[EPOCH]: 0 | [LR]: 0.001 | [TRAIN LOSS]: 0.50 | [TRAIN ACC]: 81.9% | [VAL LOSS]: 0.44 | [VAL ACC]: 84.1%\n",
      "[EPOCH]: 1 | [LR]: 0.001 | [TRAIN LOSS]: 0.41 | [TRAIN ACC]: 85.1% | [VAL LOSS]: 0.46 | [VAL ACC]: 83.9%\n",
      "[EPOCH]: 2 | [LR]: 0.001 | [TRAIN LOSS]: 0.37 | [TRAIN ACC]: 86.5% | [VAL LOSS]: 0.44 | [VAL ACC]: 84.4%\n",
      "[EPOCH]: 3 | [LR]: 0.001 | [TRAIN LOSS]: 0.33 | [TRAIN ACC]: 87.9% | [VAL LOSS]: 0.45 | [VAL ACC]: 83.8%\n",
      "[EPOCH]: 4 | [LR]: 0.001 | [TRAIN LOSS]: 0.29 | [TRAIN ACC]: 89.3% | [VAL LOSS]: 0.46 | [VAL ACC]: 84.1%\n"
     ]
    }
   ],
   "source": [
    "# Train\n",
    "trainer = Trainer(dataset=dataset, model=model, \n",
    "                  model_state_file=args.model_state_file, \n",
    "                  save_dir=args.save_dir, device=args.device,\n",
    "                  shuffle=args.shuffle, num_epochs=args.num_epochs, \n",
    "                  batch_size=args.batch_size, learning_rate=args.learning_rate, \n",
    "                  early_stopping_criteria=args.early_stopping_criteria)\n",
    "trainer.run_train_loop()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 335
    },
    "colab_type": "code",
    "id": "98MqlEQ0sfTs",
    "outputId": "3aee1562-ad04-4936-ef1b-31161d8e31c9"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA24AAAE+CAYAAAD1QEO5AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3Xd4VOed/v/3jKRRGfVeURkBAtGr\nAdGNAYNNtU1iOzjeZOONnWpnWSsuiZ04zu8XZ9dlE5e4J5sQgygG3CCI3pvpoF5Q7wX1+f4hGFs2\nzbak0Uj367q4xJQzc+sgMedznud8HoPVarUiIiIiIiIiPZbR3gFERERERETk2lS4iYiIiIiI9HAq\n3ERERERERHo4FW4iIiIiIiI9nAo3ERERERGRHk6Fm4iIiIiISA+nwk2kEw0cOJDCwkJ7xxAREek2\ny5Yt4/bbb7d3DJFeT4WbiIiIiHwt586dw8vLi/DwcI4cOWLvOCK9mgo3kW7Q2NjIE088wezZs5k7\ndy7PPvssra2tAPz1r39l7ty5zJkzh6VLl3L+/Plr3i8iItJTrFmzhjlz5jB//nzWrl1ru3/t2rXM\nnj2b2bNn84tf/IKmpqar3r9v3z5mzZpl2/bzt1988UUee+wxli5dyltvvUVbWxu//vWvmT17NjNm\nzOAXv/gFzc3NAJSXl/PAAw8wc+ZMbrvtNnbu3Elqairz58/vkHnx4sVs3ry5q3eNSKdztncAkb7g\n7bffprCwkI0bN9LS0sI999zDhg0bmDlzJs8//zxbt27F09OTDz74gNTUVMLCwq54f//+/e39rYiI\niADQ2trKJ598woMPPoiTkxPPPfccTU1NFBcX8/vf/561a9cSHBzMj370I9555x3mzJlzxfuHDh16\nzffZtm0b69atw9/fn48++oiDBw+yYcMG2traWLRoEZs2bWLBggU899xzWCwWXn75ZU6dOsV3v/td\nduzYQUlJCWfOnCEhIYELFy6Qk5PDlClTumkviXQeFW4i3SA1NZX7778fZ2dnnJ2due2229i1axe3\n3norBoOBVatWMX/+fObOnQtAc3PzFe8XERHpKXbu3MnQoUPx9PQEYNy4cWzdupXKykpGjhxJSEgI\nAM899xxOTk6sXr36ivcfOnTomu8zfPhw/P39AZg9ezbTp0/HxcUFgKFDh5Kbmwu0F3ivvfYaAIMH\nD2bLli2YTCZmz57Nxo0bSUhIYPPmzcycOROTydT5O0Ski2mqpEg3KC8vx8fHx3bbx8eHsrIyXFxc\neOuttzh8+DCzZ8/m29/+NmfPnr3q/SIiIj1FSkoKqampjBkzhjFjxvDxxx+zZs0aKioq8Pb2tj3P\n1dUVZ2fnq95/PZ///CwvL2fFihXMnj2bOXPmsGXLFqxWKwCVlZV4eXnZnnu5oJw3bx4bN24EYPPm\nzdx6663f7BsXsRMVbiLdIDAwkMrKStvtyspKAgMDgfazgi+88AJ79uwhKSmJJ5988pr3i4iI2FtV\nVRX79+9n3759HDx4kIMHD3LgwAGOHz+O0WikoqLC9tza2lpKS0vx8/O74v1OTk62674Bqqurr/q+\n//3f/42zszPvv/8+H374IVOnTrU95uvr2+H18/LyaG5uZuzYsbS0tLB161bOnz/PxIkTO2s3iHQr\nFW4i3WDatGmsWrWK1tZW6uvrWbduHVOnTuXs2bP8+Mc/pqmpCZPJxJAhQzAYDFe9X0REpCfYuHEj\nN910U4cph87OziQlJdHU1MThw4fJy8vDarXy5JNPsmrVKqZOnXrF+4OCgigpKaGsrIzW1lbef//9\nq75vWVkZAwYMwGQycebMGY4cOUJ9fT0AM2bMYM2aNQCkpaWxePFiWltbMRqN3HrrrTz99NPMmDHD\nNs1SxNHoGjeRTnbvvffi5ORku/2b3/yGe++9l9zcXObNm4fBYGDOnDm269YiIyOZP38+Li4umM1m\nnnjiCQYMGHDF+0VERHqCtWvXsnz58i/dP2vWLP70pz/x1FNPsXz5cpycnBg6dCjf/e53cXV1ver9\nS5YsYeHChYSHh7NgwQJOnz59xfe9//77WbFiBSkpKYwZM4YVK1bwy1/+kmHDhvGLX/yCFStWMGPG\nDMxmM3/4wx9wc3MD2qdLvvnmm5omKQ7NYL08MVhEREREpBcqLS1l0aJFpKamdji5KuJINFVSRERE\nRHq1F154gW9961sq2sShqXATERERkV6ptLSUmTNnUlpayv3332/vOCLfiKZKioiIiIiI9HAacRMR\nEREREenhVLiJiIiIiIj0cD1mOYCSkppv/Bp+fh5UVNR3Qpqu50hZwbHyKmvXcKSs4Fh5+1rWoCCv\nTkrTN3TG5yP0vZ+z7qKsXceR8ipr13CkrND1n5G9asTN2dlxOgU5UlZwrLzK2jUcKSs4Vl5lle7g\nSP92yto1HCkrOFZeZe0ajpQVuj5vryrcREREREREeiMVbiIiIiIiIj2cCjcREREREZEeToWbiIiI\niIhID6fCTUREREREpIdT4SYiIiIiItLDqXATERERERHp4VS4iYgIqalbbuh5v/3tb7lwIb+L04iI\niMgXqXATEenjCgousHnzRzf03F/+8peEh0d0cSIRERH5IucbedIzzzzDsWPHMBgMJCcnM2zYMNtj\nM2bMIDQ0FCen9pXC//CHPxASEnLNbbpCVmE15y7UMCDcq0vfR0Skt/njH3/P6dMnmTx5LLfcMpeC\nggv8z//8id/97ilKSoq5ePEi99//70yaNJl7772Xhx76OVu3bqGurpacnGzy8/P48Y8fZsKESfb+\nVkRERLpdWVUDh8+VsGB6/y59n+sWbvv37yc7O5uVK1eSnp5OcnIyK1eu7PCc1157DbPZ/JW26Wwf\n7sth/+livn/bYCYkhnbpe4mI9Cbf+ta9pKT8k9hYCzk5WfzpT3+hoqKcceNuYu7c+eTn5/H44//F\npEmTO2xXXFzEH/7wAnv37mbdutUq3EREpE+pvdjMxj1ZbDmUT0trGwNiA4gO9Oiy97tu4bZnzx5u\nvvlmACwWC1VVVdTW1uLp6dmp23xTC5JiOZFZztsfnCE8wEx0qEbeRMTx/PNfaRw4U9yprzk2IZg7\nZ8Tf0HMHDUoEwMvLm9OnT7J+fQoGg5Hq6qovPXfYsBEABAcHU1tb23mBRUREerDG5lY2H8xl094c\nLja2EODtyqIpcYxOCKa0tOs+D69buJWWlpKYmGi77e/vT0lJSYci7MknnyQ/P5/Ro0fz8MMP39A2\nX+Tn54Gzs9PX/T4ICvLi4W+P5uk39vHndSf440+n4uPp+rVfrzsEBTlWcelIeZW1azhSVnCsvJez\nunuYcHIydOpru3uYrrkvfH09cHV1wWx2xc/Pi6AgL9asWUNT00X++c+VVFZWsnTpUttr+PmZMZtd\n8fExExTkRUWFGRcXJ4fa3yIiIl9Va1sbu44XsnZHBpW1TZjdnFk2I57poyJwcXbCYOjcz+8vuqFr\n3D7ParV2uP3jH/+YyZMn4+Pjw4MPPshHH335AvcvbnMlFRX1XzXKl4xLDGVBUizrdmby2zf28fO7\nhuNk7Jn9V4KCvCgpqbF3jBvmSHmVtWs4UlZwrLyfz3rbTf247aZ+nf4e19oX1dUN1Nc3UFfXiItL\nAyUlNeTmFuLnF0RZWR3r1r1PQ0Oj7TUqKuo6PLeioo6mppYb3t8q8ERExJFYrVaOnC9l9bZ0Csrq\nMTkbmTchmrnjo/Fw+8rl1Nd23XcKDg6mtLTUdru4uJigoCDb7YULF9r+PmXKFM6dO3fdbbrSbZNi\nyC6s4WhaKau3ZXDn9BubHiQi0ldFR8dy9uwZwsLC8fX1BWDatBn813/9nFOnTjBv3u0EBwfz5puv\n2TmpiIhI9zqXW8l7qWmk51djNBiYOiKc2yfF4ufV/TP7rlu4TZo0iRdffJFly5Zx8uRJgoODbVMe\na2pq+OlPf8qf//xnTCYTBw4cYPbs2YSEhFx1m65mNBj43vzB/Oadg3y4L4eYUC/GDQrplvcWEXFE\nfn5+pKRs7HBfWFg4b7/9D9vtW26ZC3w2OhgX99lJsbi4eF566dXuCSsiItIN8ktqWb0tg6Np7YNR\nowcEsXhqHGEB5uts2XWuW7iNGjWKxMREli1bhsFg4MknnyQlJQUvLy9mzZrFlClTuOuuu3B1dWXw\n4MHMmTMHg8HwpW26k4ebMz9aMpSn3z7IG5tOExZgJiq4ewpHERERERFxTOXVDazdkcmuEwVYrTAg\n0oc7psdjifCxd7Qbu8btkUce6XA7ISHB9vfly5ezfPny627T3cICzHxv/mBeSjnOi6s/5Yn7xuLp\n7mLXTCIiIiIi0vPUXmxm095sNh/Mo6W1jYggM0unWhhmCejypiM3qvuuprODUQOCuG1iDO/vzuKV\n9Sf52R3DMRp7xo4XEZHer66ujhUrVlBVVUVzczMPPvggZrOZZ599FhcXF0aPHs3Pf/5ze8cUEemz\nmppb2XIoj417sqlvbMHf25VFk+OYkBja4+qGXl24ASyYHEt2UQ2fppeRsj2DpdMs9o4kIiJ9xJo1\na4iNjeXhhx+mqKiI5cuXYzKZ+OMf/0h8fDzJyckcPnyYUaNG2TuqiEif0tZmZdfxAtbuzKSiphGz\nmzN3To9n5uj21v49Uc/sld+JjAYD/37bYIL93Nm0N7vTF7YVERG5Gj8/PyorKwGorq7Gz8+PkpIS\n4uPbm7skJSWxa9cue0YUEelT2lv7l/DEG/t584Mz1F5s5tabovn9AxOYM75fjy3aoA8UbgAebi78\naPFQXE1OvLHxNHklXbeiuYiIyGXz5s3jwoULzJo1i3vuuYcVK1YQGRnJgQMHsFqt7N69u8PyOSIi\n0nXO51Xyu78d5sXVxykoq2PK8DCe/cEElk6z4OHW83th9PqpkpdFBHnyvXmD+N81J3hp9XEev28M\nZgf4BxIR6SmWLr2NTZs2Xv+JYrNu3TrCw8N5/fXXOXPmDMnJyTz77LP89re/xcnJiYSEBGprr30y\n0c/PA+dOOgPsSIufK2vXcKSs4Fh5lbVrdEbWnMJq3tl0mn0nCwG4aUgo37l1MFEhnb8funLf9pnC\nDWD0wGDmTYhm455sXl1/ip8sHdbjLjoUEZHe4/DhwyQlJQHtHZmLi4uxWCy8/fbbAPzjH/+gurr6\nmq9RUVHfKVkur8HnCJS1azhSVnCsvMraNb5p1vLqBtbtzGTn8fbW/vGRPtw5LZ74yPbW/p29Hzpj\n316r8OsTUyU/b9HkOIbE+XM8o4y1OzPtHUdExO7uv/9uCgvbz0IWFhbw3e9+m//8z5/yox/9gO9/\nfzmnTp2wc0LHFR0dzbFjxwDIz8/HbDbz2GOPcebMGVpbW1m3bh3Tpk2zb0gRkV6mrqGZ91LTePTV\nvez4tICwADM/XjKMR+8eZSvaHFGfGnEDMBoN/OD2RJ566wAbdmcRHeLF6IFB9o4lImI3U6ZMZ9eu\n7SxZcic7dmxjypTpWCz9mTJlGocOHeBvf3ub3/72/7d3TId01113kZyczD333ENLSwu/+tWvMJlM\nPProowDMnz+fAQMG2DmliEjv0NzSypZD+Wzck0VdQwt+Xq4snBzLpCFhvWKWXZ8r3ADMbi78aPEw\nfvPuQf6y8RShAWOICDTbO5aICClpGzhSfLxTX3Nk8FAWx8+/6uNTpkznpZf+hyVL7mTnzm089NDP\n+Mc/3uXvf3+X5uZm3NzcOjVPX2I2m3n++ee/dP+aNWvskEZEpHdqa7Oy+0Qha3dmUF7diIerM3dM\ntzBzVCQml57bJfKr6nNTJS+LDPbk/lsH0djUykurP6W+ocXekURE7CIuzkJZWQlFRYXU1NSwY0cq\ngYHB/PnPr/PII/9l73giIiJXZLVaOZpWypNv7OeNTaeprmtm7vh+/P4/JjB3fHSvKtqgj464XTZu\nUAjZhTV8sC+H194/yY+WDsNocPxhVBFxXIvj519zdKyrTJiQxKuv/onJk6dSWVmBxdIfgG3bttLS\nohNbIiLSs6TlV7Fqaxrn8qowGCBpWBgLk2Lx9+69s0T6dOEGsGSqhZyiGo6ll7F+ZyYLJ8fZO5KI\nSLebOnU6DzxwP2+99XcaGi7ym988ydatm1my5E42b/6YjRvX2zuiiIgIBWV1rN6WweFzJQCMiA9k\nydQ4IoI87Zys6/X5ws1oNPCDBUN46q0DrN+VRXSoFyP7q1mJiPQtgwYlsm3bPtvtv/1tle3vSUlT\nAZg373bMZjP19Y7RRlpERHqPippG1u3MZMenF9pb+0f4sHSahQFRvvaO1m36fOEG4OnuwkOLh/LM\nu4d47f1TPL58DGEBalYiIiIiImJP9Q3NfLAvh08O5NLU0kZYgAdLp1oY0T8QQx+7xEmF2yX9Qry4\n79YEXl1/ipdSjvPYd8bg7qrdIyIiIiLS3ZpbWlm7LY1/fHyWuoYWfD1NfHtyHJOGhuJk7Jv9FVWZ\nfM5Ng0PJLqzho/25/GXDKR5cPFTNSkREREREuklbm5U9JwtZuyODsupG3F2dWTrNwszRkbj2si6R\nX5UKty9YOs1CTlEtR86XsmF3FrdPirV3JBERERGRXs1qtXI8o4xVqenkldTh7GRk0bR4pg8Pw9Pd\nxd7xegQVbl/gZDTywIJEnnrrIOt2ZNIvxIsR8YH2jiUiIiIi0iulX6hi1dZ0zuZWYgAmDQ1lYVIc\nCfFBlJSoIdZlKtyuwMvD1N6s5K+HeO39kzy+fCyh/h72jiUiIiIi0msUlNWRsj2DQ2fbW/sPtwSw\nZKqFyODe39r/61DhdhXRoV7cNyeB1za0Nyv55b2j1axEREREROQbqqxtZP3OTLYfK6DNasUS7s3S\naRYG9vOzd7QeTZXINUwYEkpWYQ2fHMzljY2n+eGiIX2u7aiIiIiISGeob2jhw/3ZfHwgl6bmNkL9\nPVgy1cKoAX2vtf/XocLtOu6YbiG3uIZD50rYtDebeRNi7B1JRERERMRhNLe0sfVIPht2Z1F7sRkf\nTxPfmhlL0rCwPtva/+tQ4XYdzk5GHlgwhKfePkDKtgz6hXgxNC7A3rFERERERHq0NquVfSeLSNme\nQVl1A+6uTiyZGsfNY6L6fGv/r0OF2w3wNpt4cNFQfvfXw7yy7iRP3DeGYD81KxERERER+SKr1cqJ\nzHJWpaaTW1yLs5OBW8ZGMX9ijFr7fwMq3G5QbJg335k9kDc2nebFS81K3EzafSIiIiIil2UWVPPe\n1jTO5LS39p84JJSFk2MJ9HG3dzSHp8rjK0gaFkZ2YQ1bDufxxqYz/MeCRF1IKSIiIiJ9XlF5Pau3\nZ3DwTDEAwy619o9Sa/9Oo8LtK7prZjy5xTUcPFPMh6FezL0p2t6RRERERETsoqq2kfW7sth29AJt\nViuxYd7cMc1CQrRa+3c2FW5fkbOTkf9YNJSn3jrAqm3pRIV4MiRWzUpEREREpO+42NjCh/ty+OhA\nDk3NbYT4ubNkqoXRA4M0I62LqP/m1+BjNvHDRUNwMhp4Zd1Jiisv2juSiIiIiEiXa25p45ODuax4\neQ/v787CzeTMvbMH8vT3xjMmIVhFWxfSiNvXZAn34d5bBvLmB2d4aXV7sxJXk9qaioiIiEjv02a1\nsv9Ue2v/0qoG3ExOLJoSxy1jonQM3E1UuH0Dk4eHk1VYw9Yj+bz14Rn+/bbBOssgIiIiIr2G1Wrl\nZFY5q7amk1Nci5PRwKwxUcybGI23h8ne8foUFW7f0Ldu7k9ucS37ThURE+rF7HH97B1JREREROQb\nyyyoZlVqOqezKzAAExJDWDg5jiBftfa3BxVu35Czk5EfLhrCr986wD+3phEV7MngGH97xxIRERER\n+VqKKupZsz2D/afbW/sPifNn6VQL/UK87Jysb1Ph1gl8PV15cNFQfv+3w7y87iRPLB9DoM5EiIj0\neXV1daxYsYKqqiqam5t58MEHqa+v54033sDFxYWQkBB+97vfYTJpupGI2F9VXRPv78pk29ELtLZZ\niQn14o5pFgZpUKJHUOHWSeIjfLj7lgG88+FZXko5zqP3jsbVRRdqioj0ZWvWrCE2NpaHH36YoqIi\nli9fTl1dHZs2bcLLy4vHH3+cTz75hHnz5tk7qoj0YRcbW/hofw4f7c+lsbmVYF93Fk+NY6y6RPYo\nKtw60bQREWQV1LD92AXe/vAM35+vZiUiIn2Zn58fZ8+eBaC6uho/Pz9cXFyorq7Gy8vLdp+IiD20\ntLax7egF1u/KpKa+GW8PF+6YbmHK8HCcnbRqWE+jwq2T3T1rAPkltew9WURsqDezxkbZO5KIyNdi\ntVopa6ggoyqLjKpssqqyGRY+iFsjZ9s7msOYN28eKSkpzJo1i+rqal555RUaGxtZtGgRXl5eDB48\nmIkTJ9o7poj0MW1WKwdOF5OyPZ2SygZcTU4sTIrllnFRuJlUHvRU+pfpZC7ORn64aChPvXWAlf9q\nb1aSEK2zqSLS87W0tZBbc4HMqizSq7LJrMqiqqnG9riL0ZmxzsPsmNDxrFu3jvDwcF5//XXOnDnD\no48+SktLC6tWrSIqKoqf/vSnbNmyhZkzZ171Nfz8PHB27pyp90FBjtNYQFm7hiNlBcfK6yhZj54r\n5q2Np0jPq8LZycD8pFjuunkgvl6u9o52RY6yXy/ryrwq3LqAn5crP1w0hP/v/47w53UneGL5WAJ8\n3OwdS0Skg9rmOjKrssmoyia9Moucmlya21psj/uYvBgZNJQ4n2jifGOI9AwnLMSPkpKaa7yqfN7h\nw4dJSkoCICEhgfT0dKKjo+nXr33pmAkTJnDixIlrFm4VFfWdkiUoyMth/u2UtWs4UlZwrLyOkDW7\nsIZVqWmczKoAYPzgEBZNjiXYz4PmhiZKGprsnPDLHGG/fl5n5L1W4afCrYv0j/Tl2zf3592Pz/HS\nmuM8evcoTGpWIiJ2YrVaKaovIaMq2zb1sai+2Pa4AQPhnqFYfGKI9YnG4hODv5ufrtP9hqKjozl2\n7BizZ88mPz+f0NBQqqqqKC8vx9/fn+PHjzN27Fh7xxSRXuyLrf1HDAhiwcQYokMdayRLVLh1qWkj\nI8gsrGHnpwW8+9FZ7p83SAdBItItmlqbyanJI6Myi/SqLDKrs6lr/mzkxs3JlQS//rbRtBjvfrg7\na2ZAZ7vrrrtITk7mnnvuoaWlhV//+tfU1dXxwAMPYDKZiIyMVEdJEekSVbWNrN+dxfZLrf2jQ71Y\nOs3CtLHRDjWKJZ9R4daFDAYD997S3qxk14lCYsK8mTk60t6xRKQXqmqs7jCalluTT6u11fZ4gJsf\ng/wHYPGJIc4nhnDPUIwGdQzramazmeeff/5L99988812SCMifcHFxhY+3JfDxwcutfb3c2fxlDjG\nJARj1ACCQ1Ph1sVcnJ148FKzkn9sOU9kkJmB/dSsRES+vjZrGxdqCzsUamUN5bbHjQYjUV4R7aNp\nPjHE+UTj6+pjx8QiItLVmlvaSD2Sz/u7s6i92Iy32cSd0y1MVmv/XkOFWzfw93bjPxYO4Q//OMqf\n157gifvGOlyHHBGxn4aWBrKqc9unPFZlk1mVQ0Nrg+1xD2d3hgQkXCrSYoj2jsTkZLJjYhER6S5t\nbVb2nipk7Y5MSqsacDM5sWhyLLPGqrV/b6N/zW4ysJ8fd82I5/82n+d/15zgDz+ZYu9IN6TN2kZN\nUx0BVrO9o4j0CVarlfKGyksjae2jafm1BVix2p4T4hHECJ8hl6Y9RhPsEaRpjyIifYzVauV4Rjmr\nUtPJK6nF2cnArDFRzJ8YjZeHTt71RircutHM0ZFkF9aw60QhL6d8yrLplh7TrMRqtVLVVE1BbREX\n6gq5UFdIQW0RBfVFNLU2YTZ5EOcdjcUnlnjfOPp5ReBkVJdMkW+qta2VtLIsDuWeap/6WJlFVVO1\n7XFno3OHKY9xPjF4mnQiRUSkL0u/UMWqremcza3EAEwcEsrCpFgCfd3tHU26kAq3bmQwGLh39kDy\nSuv4ZH8OoX7uTB8Z0e05apvrKKgt5EJd0aUCrZCCuiLqWy52eJ6zwYkQczABbv4UXSzieOlpjpee\nBsBkdCHWJ5p43/ZCLsa7HyYnl27/XkQcTV1zvW3ttIyqLLKqc2lua7Y97mXyZETQEFuhFuUVgbNR\n/1WLiAgUlNWRsi2DQ+dKABhmCWDJVAtRwZ52TibdQUcD3czk4sRDi4by9DsH+b9PzhEZZKZ/pG+X\nvFdDSwMFl4uzuiLbaFp1U8cWsAYMBHsEMsAvnnBzCGGeoYSbQwhyD7SNqgUFeXE+L4+0ysxLfzI4\nW5HG2Yo0AJwMTkR7R14akYvF4huDu7PO+kjfZrVaKb5YSkZllq1QK/zC2mlh5hASQ/sTZgrH4htD\ngJt/jxmJFxGRnqGippF1OzPZ+WkBbVYrlnBvlk6zqOFdH6PCzQ4CfNxY8Z0xPP7yHv60pr1ZiZ+X\n69d+vabWZorqi9uLtNpCCuraR9PKGyq+/N5ufgwJGESYOYRwz1DCzKGEegThcgOjZb6uPowJGcGY\nkBFA+8hdemUW6ZeKuazqXDKqsvkkJxUDBiI8w2wjcvG+sXiZdDZIerfm1maya/LIrMq2NRKpba6z\nPW5yMjHQL942mhbr0w93Z3eCgry0po6IiHxJXUMzm/Zms/lgHs0tbYQFeLBkqoWR/QN1kq8PuqHC\n7ZlnnuHYsWMYDAaSk5MZNmzYl57z3HPPcfToUd5991327dvHT37yE/r37w/AgAEDePzxxzs3uYMb\nFh/EnTPi+ceW8/xpzXH+89ujcHG+dnOB1rZWSi6Wtk9xtBVohZTUl3VoXADgbfIiwa8/YZ4hhJvb\nC7QwczBunbjArqeLmeFBiQwPSgSgoaWRzOps24hcVnUuebUXSM3bBbQ3VLg8IhfvG0eAu84SiWOr\nbqrpMJqW84W10/xcfRkTMoJYn2gsPjGEm0N1baiIiFxXU3MrWw7nsWlPNnUNLfh5ubIgKZZJQ0Nx\nMqoZVV913cJt//79ZGdns3LlStLT00lOTmblypUdnpOWlsaBAwdwcfls1GbcuHG88MILnZ+4F5k1\nJpKswmr2nizi/zafY/mcBKC9k2N5QwUXLl2HVlBXyIXaQorrS2j53EEhtLcBv7yYbrg5pL1A8wzB\n06X7mxe4ObsyyH8Ag/wHANBStByMAAAgAElEQVTc1kJ2da5tRC6jKovdBfvZXbAfaD+obS/i2v+E\neATr7JH0WG3WNgrqijqsnVZ6scz2uNFgJNIzvEMjET+3rpkGLSIivVNrWxu7jxeydmcmFTWNeLg6\nc8c0CzNHR2Jy0Ym/vu66hduePXu4+eabAbBYLFRVVVFbW4un52fT3p599ll+9rOf8dJLL3Vd0l5q\nwfQwsmrS2VW4i+Kde7C61lJQV0jT55oVQPsUqwivcMLNHQs0H5N3jy12XIzOtqJsNu0jhvl1Bbbr\n5NIrMzlQdIQDRUeA9hE8y+cKuUjPcLU4F7tpaGkkqzrHNu0xqzqHiy2frZ3m7uxOYkCCrVCL9o7C\nVWuniYjI12C1Wjl6vpTV2zO4UFqHi7ORueP7ceuEaMxuav4m7a5buJWWlpKYmGi77e/vT0lJia1w\nS0lJYdy4cUREdOyOmJaWxgMPPEBVVRUPPfQQkyZNuub7+Pl54Oz8zc8k9NSFrasba8mtuvDZn0/b\nv9Y1X4RQcAGymsCp2YlInzCivMOI8gknyiecfj7hBJr97V7EdMa+DcWX0QwC2v+Tyq8p5HRxGqdL\n0zhdcp5jJSc4VnICAHdnNwYGxjEoqD+DguKx+Eff0LV4nZW1uyhr1/kqeUvryjlbls7ZkgzOlqaT\nVZWH1frZFOQwz2DGBY5gYKCFgYFxRHiHdurvpCPtW0fKKiLS053LrWRVajpp+VUYDDB5WBgLkmLx\n9+68y1ukd/jKzUk+fyBTWVlJSkoKb775JkVFRbb7Y2JieOihh5g7dy65ubl85zvf4eOPP8ZkuvrZ\n6IqK+q8a5Ut6wgX+F1saKLS12f9sTbSaptoOzzMYDAS7B9Lft72TY2u9J+u3lOLp5Mu/3zceX8/P\nNSu5CGUX67Cnrtq3rngywmcEI3xGgAXKLlaQVpnRPiJXlcnRwlMcLTwFtK9nFeMdZWt2EusdjZvz\nl5u69ISfgxulrF3nWnlb21rJq73QYdpjZWOV7XFngxOx3v0uTXlsn/bYoblOE5SVdt7vpCPt287I\nqsJPRATySmpJ2ZbB0bRSAEb2D2TJVAvhgVqrU67suoVbcHAwpaWlttvFxcUEBQUBsHfvXsrLy7n7\n7rtpamoiJyeHZ555huTkZG699VYA+vXrR2BgIEVFRURFRXXRt9H9LndyvHBpDbQLl65Dq2is/NJz\nL3dybO/i2N4sJDE6jqryhg7PM9Xl8M+tafxp7Qn+81sjcXbqe9MEA9z9CHAfzfiw0UB784f0yqzP\nirnKLNIqM4H2a4qiPCMuLT/QvgSBPa7tk56vvrmezOocWyORrOqcDtORPV3MDA9MJM738tppkbho\n7TQREekCZVUNrN2Rwe4ThViBAZE+LJ0eT3yEj72jSQ933SOTSZMm8eKLL7Js2TJOnjxJcHCwbZrk\nnDlzmDNnDgB5eXk8+uijJCcns379ekpKSvi3f/s3SkpKKCsrIyQkpGu/ky7S2tZK8cXSDm32C67S\nydHnUifHywXatTo5ti9W3bFwmz0uiqzCavafLubvW85z7y0Du/JbcwjeJi9GBg9lZPBQAOqbL5JR\nlWUbkcuuziO7JpctudsBCDOHMDR0IBFukcT7xuLrqv8E+5qWthbKGio4mXmCY3lnyKjKpqCuqMNz\nwswhxPnEYPGJIdYnmiD3gB57raiIiPQOtReb2bA7i38dzqeltY3IIDNLploYZtFnkNyY6xZuo0aN\nIjExkWXLlmEwGHjyySdJSUnBy8uLWbNmXXGbGTNm8Mgjj7Blyxaam5v51a9+dc1pkj1Bm7WNsosV\nlxarLrStiVZUX9KhvTd0XSdHg8HAd+cO4kJpPVsP5xMT4sXk4eHf6DV7Gw8Xd4YEDmJIYPt1ck2t\nTWRV59pG5DKrsvk4fbvt+YFu/sT7xtmanugA3bG1WduoaqymorGKioZKKhorqWyooqKxkopLX2ua\najucVDEZXRjga7GNpsV698PDxcOO34WIiPQljU2tfHIwlw/2ZXOxsZUAb1cWTo5jQmIoRqOOSeTG\n3dBcoEceeaTD7YSEhC89JzIyknfffRcAT09PXn755U6I1/msViuVjVW2kbPL16EV1hVdsZNj5Oc7\nOXqGEm4Oxdvk1WUH/64mJx5aMpSn3zrAux+fJSLIk7hw7y55r97A5GRigJ+FAX4WoH2EtMa5ggOZ\nJ0ivyiStMou9hQfZW3gQaB/B+/yi4GHmELs3fZF2VquV2uY6W0FWYSvIKm2FWlVTNW3Wtitu72xw\nwtfVh3jfWPzcfBkcZiHEKYwIzzCtnSYiIt2upbWNnZ8WsG5nJlV1TXi6u7BsRizTR0Xg0gkN+aTv\n6dUXcdQ01dquPyv43Jpon2/pDe0HfCHm4EsFWqht0Wo/N1+7HNQH+7rzgwWJ/Pc/j/G/a47zxH1j\n8TH37BHLnsLJ6ET/gFh82wKZxTTb2luXFwVPq8zkcPGnHC7+FPhs9PRyMdfPK0IH+V3AarVyseVi\nh5GyLxZmlY1VtLS1XHF7AwZ8XL2J9orCz80HP1df/Nx88XP1af/q5ouni7nD76sjNfwQEZHew2q1\ncuhsCau3pVNUcRGTi5H5E2OYM64fHm69+tBbuliv+empaarlWPoxzhVmtRdotYXUNH+hkyMGgj2C\nSPDr334N2qURtCD3gB53sD4kNoAlUy2sSk3nz2uO80gfbVbyTRkNRiI820ddpkZOxGq1UnKxzLaO\nXFplBifKTnOi7DTQPq0u1ifatpZcjHc/TFqb67oaWhqpvMooWUVj+31NrU1X3d7L5Gk7WWIrxmxf\nffE2efW431EREZEvOp1dwarUNDILanAyGpg+MoLbJsV07BYu8jX1msLtn+fW2kZRAALc/Bni3bGT\nY4hH0A2vA9YTzB3fj6yCag6eLWHlv9K4e9YAe0dyeAaDgWCPQII9ApkYPhaAysaqDouCn61I42xF\nGgBOBif6eUXaCjmLbwzuzu72/Ba6XXNrMxWNVRQVXSCrqOCKhdnFlotX3d7s7EGQe8CXR8kuffVx\n9VEHRxERcWgZ+VW8tuZTTmSWAzA2IZjFU+II8dc11dJ5es3R0vy42dwUMwJzmzehHiFXXN/L0RgM\nBu6fN4iC8nq2HMojJtSLSUPD7B2r1/F19WFMyAjGhIwAoLa5jvTKrEsjcplk1+SSWZ3NJzmpGDAQ\n4RlmW4Ig3jcWb5PjrknV2tZKVVP1l6ctXprOWN5QSW3z1dcrc3Uy4efmR4x31KXCrOM0Rl83X1w1\nYikiIr1UceVF1m7PYO+p9u7Fg6L9WDrNQmyY+hNI5+s1hVuIRxBDguJ63TUtbiZnHlo8lKffOsjb\nH54lIshMTKj+M+hKni5mhgclMjwoEWifBphZnW0bkcusziGv9gKpebsACPYIJN4nzjYq5+/m1yM6\nV7ZZ26hpqr3y9WSXvlY1Vn9pWYvLXIzO+Lr62KYwRvgHYWr16DCF0d3ZrUd8ryIiIt2puq6J93dn\nkXokn9Y2K5ZIHxZOiiUx1t/e0aQX6zWFW28W4ufBv9+eyPPvHeOllPZmJd4eGsXoLm7OrgzyH8Ag\n//apqs1tLWRX59pG5DKqsthdsJ/dBfsB8HP1tY3I9feNJcQjuNOLG6vVSl1z/RWmLbYXaZWNlVQ2\nVn9pKYvLjAYjvq4+xPlE24owX9toWftXTxdzh9xq9iEiIn3dxcYWPj6Qy4f7c2hsaiXY151FU+K4\ndbKFsrLa67+AyDegws1BDLMEsGhKHCnbM3h57QkeXjYCJ6OaldiDi9HZNro2m/bphvl1BR2ukztQ\ndIQDRUeA9hG8y9Mq431ib6g9/cWWi1dt9HF5tKz5C8tXXGbAgLfJiyiviA7Xk/leKtD83HzwNnlp\nGQQREZEb1NLaRuqRfN7fnUVNfTPeHi4snWph6ohwnJ2MWo9NuoUKNwcyb0I02YU1HDpXwntb01k2\ns7+9IwntSxD084qkn1ckM6ImY7VaKaovthVyaZWZHCs5wbGSEwC4Oble6lwZR1RtMDklhZRfHi27\nVJg1tDZe9f08XcyEmoM7XlP2ucLM19VbHRhFREQ6QZvVyv5TRazZkUFJZQOuJicWJsVyy7go3Ew6\njJbupZ84B3K5WcmFsjo+PpBLdIgXE4aE2juWfIHBYCDUHEKoOYSkiJsAKLtYYVtHLr0qk9Pl5zhd\nfg4yOm7r7uyGv5vfZ9MWOzT88MHX1ReTA3VGFenr6urqWLFiBVVVVTQ3N/Pggw/y6quv2h4vLi5m\n0aJFPPDAA3ZMKSJfZLVaOZlZzqrUdHKKa3EyGrh5dCTzJ8bgrbV1xU5UuDkYd1dnfrRkGE+/fYC3\nPjxDeKCZ6FDH7WrYVwS4+xHgPprxYaMBqG6qIb0yCye3Npyb3WzTGd2c3eycVEQ605o1a4iNjeXh\nhx+mqKiI5cuX8+GHH9oe/973vseCBQvsmFBEviizoJpVqemczq7AAExIDGHh5DiCfPvWckDS86hw\nc0Ch/h58/7ZEXlj16aVmJWPwUrMSh+Jt8mJk8FA1/BDp5fz8/Dh79iwA1dXV+Pn52R7bvXs3MTEx\nhIVpmReRnqCwvJ6U7RkcPFMMwNC4AJZMjaNfiE6QS8+gws1BjYgPZGFSLGt3ZvLyupP8/K7halYi\nItLDzJs3j5SUFGbNmkV1dTWvvPKK7bF33nmH5ORkO6YTEYDK2kbW78xk+7EC2qxWYsO8uWOahYRo\nv+tvLNKNVLg5sPmTYsguquHI+VJWp2Zw54x4e0cSEZHPWbduHeHh4bz++uucOXOG5ORkUlJSKCoq\nor6+nn79+l33Nfz8PHB27pyGQ0FBjjNyoKxdw5GyQtfmrbvYzOqt51m/I4PGplYigsx859bBTBga\n9rWW8XGkfausXacr86pwc2BGg4HvzR/M028f5MP9OUSHejF+cIi9Y4mIyCWHDx8mKSkJgISEBIqL\ni2ltbWXbtm3cdNNNN/QaFRX1nZLFkaZmK2vXcKSs0HV5m1ta+dfhfDbszqKuoQUfTxPLZsSTNCwM\nJ6OR0tKvvh6bI+1bZe06nZH3WoWf5tY5uPZmJUNxMznx5qbT5BQ5zg+3iEhvFx0dzbFjxwDIz8/H\nbDbj5OTE8ePHSUhIsHM6kb6lrc3KruMFJL+6l5X/SqPNCkumxvHsDyYwdUSELjmRHk8jbr1AWICZ\n788fzIspxy81KxmLp7taxouI2Ntdd91FcnIy99xzDy0tLfzqV78CoKSkhICAAPuGE+kjrFYrx9LL\nWL0tnfySOpydjMwZ149bJ0TreEkcigq3XmLkgCBunxTD+l1ZvLL+JD+7YzhG41efny0iIp3HbDbz\n/PPPf+n+l19+2Q5pRPqetLwq3ktN43xeFQYDJA0NY+HkWPy9tfyOOB4Vbr3I7UmxZBfWtJ9V2p7O\nHdPUrERERET6nvzSOlK2pXPkfCkAI/sHsnhKHBFBnnZOJvL1qXDrRYwGA9+/rb1ZyQd7c4gO8WLc\nIDUrERERkb6hvLqBtTsz2XW8AKsV+kf6sHSahf6RvvaOJvKNqXDrZTzcXHhoyTB+885B3th0mvAA\nM5HBOrskIiIivVftxWY27c1m88E8WlrbiAg0s2SqheHxAV+rtb9IT6TCrReKCDTzvXmD+N81J3gp\n5TiP3zcGs5suvhUREZHepbG5lS2H8ti0J5v6xhb8vV1ZmBTHxCGhutZfeh0Vbr3U6IHBzJ8YzYbd\n2by6/hQ/WTpM/4GJiIhIr9Da1sbOTwtYtzOTytomzG7O3Dk9npmjI3DppAXrRXoaFW692MKkOLIL\nazmeUcbanRksnmKxdyQRERGRr81qtXL4XCmrt6VTWF6PydnIvAnRzB3fDw/NLpJeToVbL2Y0Gvj3\n2wfz9FsH2bA7m+gQL0YPDLZ3LBEREZGv7GxOBatS00m/UI3RYGDaiHBumxSLn5ervaOJdAsVbr2c\n2c2Fh5YM5bfvHOIvG08TGmAmItBs71giIiIiNyS3uJbV29L5NL0MgDEDg1g0JY6wAB3PSN+iwq0P\niAzy5P55g/jz2hO8tPpTHl8+RtMJREREpEcrKq/n9fdPsfdkIVYgoZ8vS6fFExfube9oInahwq2P\nGJsQTNZN/fhgbw6vvn+KHy8dhlHtcUVERKSHqalvYsPubLYeyaeltY2oYE/umGYhMdZfrf2lT1Ph\n1ocsmWIhp6iWT9PLWL8zk4WT4+wdSURERASAxqZWPj6Yywd7s2loaiXE34MFk2IYNzhEJ5tFUOHW\npxiNBn5weyJPvXWA9buyiA7xYuSAIHvHEhERkT6sta2NHZda+1fVNuHp7sK3b45j6awEKivq7B1P\npMcw2juAdC9Pdxd+tGQYJhcjr204RUGZ/kMUERGR7me1Wjl0toTH/7Kfdz48y8XGFm6bGMPvH5jA\nzWOicHHWYarI52nErQ+KCvbku3MH8cr6k7y4+jiPLx+Du6t+FERERKR7nMut5L3UNNLzL7X2HxnB\n7ZNi8PVUa3+Rq9HReh81fnAI2YU1fLg/h79sOMWDi4dq/riIiIh0qfzSOlanpnM0rRSA0QODWKzW\n/iI3RIVbH7ZkWhzZRTUcOV/Kht1Z3D4p1t6RREREpBcqr25g7c5Mdh0vwGqFAZE+3DE9HkuEj72j\niTgMFW59mJPRyAMLEnnqrYOs25FJvxAvRsQH2juWiIiI9BL1Dc1s3JvN5oN5NLe0ERFoZsk0C8Mt\nAWrtL/IVqXDr47w8TDy0eCjP/PUQr71/kseXjyXU38PesURERMSBNbe0suVQPhv3ZFHX0IKflysL\nJ8cyaUgYRqMKNpGvQ4WbEB3qxX1zEnhtwyleXP0pj31HzUpERETkq2trs7LnZCFrd2RQVt2Ih6sz\nd0yzMHN0JCYXJ3vHE3FoOjoXACYMCSWrsIZPDuby+sbT/HDREDUrERERkRtitVo5nlHOqtR08kpq\ncXYyMmd8P269KRpPdxd7xxPpFVS4ic2dMyzkFtdw+FwJm/ZkM39ijL0jiYiISA+XWVDNe1vTOJNT\niQGYNDSUhUlxBPi42TuaSK+iwk1snIxGHlg4hKfeOsCa7Rn0C/FimCXA3rFERESkByqqqGf1tgwO\nnikGYJglgKVTLUQGe9o5mUjvpMJNOvC+3Kzk3cO8uv4kj983hhA/NSsRERGRdlV1Tazflcn2oxdo\nbbMSG+bNHdMsJET72TuaSK+mwk2+JCbUm+VzBvL6xtO8lHKcX9472t6RRERExM4uNrbw0f4cPtqf\nS2NzKyF+7iyZamH0wCC19hfpBirc5IomDQ0jq7CGLYfyeGPjaZ74/gR7RxIRcTh1dXWsWLGCqqoq\nmpubefDBBxkxYgQ/+9nPqKqqIiQkhD/+8Y+YTCZ7RxW5qpbWNrYdvcD7uzKprm/G22zizukWJg8P\nx9nJaO94In2GCje5qrtmxJNbVMPBsyU89fo+Fk6KITzQbO9YIiIOY82aNcTGxvLwww9TVFTE8uXL\nmTFjBklJSdx333289NJLnDlzhmHDhtk7qsiXWK1WDpwpJmV7BsUVF3E1ObEwKZZbxkXhZtIhpEh3\n02+dXJWzk5EfLhrKn9ee4ODpIg6fKWbKiHAWJMXiY9bZYRGR6/Hz8+Ps2bMAVFdX4+fnx9atW/nr\nX/8KwEMPPWTPeCJXdTq7glWpaWQW1OBkNDBzVCS3TYrBW5//Inajwk2uydts4j+/PZKM4jr+su4E\nqUfy2XOykFtviuaWsVG4ajFNEZGrmjdvHikpKcyaNYvq6mpeeeUVvv/97/P3v/+d3bt3Ex8fz2OP\nPaapktJj5BbX8l5qGicyygEYNyiYRVPi1KhMpAdQ4SbXZTAYuGlIGNGBHmw/doG1OzJZsz2D1CP5\nLJocx8QhoRiNuihZROSL1q1bR3h4OK+//jpnzpwhOTmZxsZGJk2axEMPPcRjjz3Ge++9x913333V\n1/Dz88DZuXNOkgUFeXXK63QHZe0aV8taXF7PXz88TerhPKxWGBYfyH3zB9M/yr6dInvDvu2JlLXr\ndGVeFW5yw5ydjMwYFclNg0P5YF82Hx/I5Y1Np/nkYC53To8nMdbf3hFFRHqUw4cPk5SUBEBCQgLF\nxcWEhoYycuRIACZNmsS+ffuu+RoVFfWdkiUoyIuSkppOea2upqxd40pZay82s2F3Fv86nEdLq5Wo\nYE/umGYhMdYfg8Fg1+/N0fdtT6WsXacz8l6r8FPhJl+Zh5szS6ZamD4ygpTtGew5UchzK48yJM6f\nO6fHExmkhTdFRACio6M5duwYs2fPJj8/H7PZzPjx49m7dy833XQTJ0+eJDY21t4xpQ9qam7lk4O5\nbNqbw8XGFgK83Vg8JY7xiSEY1dpfpEdS4SZfm7+3G9+bP5hZY6L459b2+fAnM/czeVgYCyfH4evp\nau+IIiJ2ddddd5GcnMw999xDS0sLv/rVrxg4cCCPPPIIL7zwAoGBgfzwhz+0d0zpQ1rb2th1vJB1\nOzOpqGnE7ObMshnxTB8VgUsnTckVka5xQ4XbM888w7FjxzAYDCQnJ1+xbfFzzz3H0aNHeffdd294\nG+kdokO9eGTZCI5nlPHPrelsP1bA3lNFzBnXjznj+6llsIj0WWazmeeff/5L97/xxht2SCN9mdVq\nZd+JAt54/yQXSuswORuZNyGaueOj8XDT57SII7jub+r+/fvJzs5m5cqVpKenk5yczMqVKzs8Jy0t\njQMHDuDi4nLD20jvYjAYGGYJJDHWnx2fFrB2Rybrd2Wx7egFFk2JI2lomBqYiIiI2EFaXhXvpaZx\nPq8KgwGmDA9jQVIcfl6aGSPiSK5buO3Zs4ebb74ZAIvFQlVVFbW1tXh6fnYd07PPPsvPfvYzXnrp\npRveRnonJ6ORaSMiGD8ohI/25/Dh/hze+uAMnxzI5Y7p8QyNa7/YWURERLpWQVkdq1LTOXK+FIDx\niaHMnxBNRKDZzslE5Ou4buFWWlpKYmKi7ba/vz8lJSW2IiwlJYVx48YRERFxw9tcSWe1O3aklqGO\nlBW+et7vR/qxeOYA/vbhGTYfyOF/3jvGiP5BfPe2ROIifLooZTtH2rfK2nUcKa+yikhnqahpZP2u\nTHYcK6DNaiU+woel0yxMGhXlUB36RKSjrzyp2Wq12v5eWVlJSkoKb775JkVFRTe0zdV0RrtjR2oZ\n6khZ4Zvl/daMeJKGhPLe1jSOni/hp39MZeKQUBZNicPf262TkzrWvlXWruNIeftaVhV+Il2jvqGF\nD/dn8/H+XJpa2ggL8GDpVAsj+gdqtotIL3Ddwi04OJjS0lLb7eLiYoKCggDYu3cv5eXl3H333TQ1\nNZGTk8MzzzxzzW2kb4oK9uTnd43gRGYZ//xXOrtOFLL/TDGzx0Uxd3w07q66MFpEROTraG5pY+uR\nfDbszqL2YjO+nia+PTmOSUNDcTIa7R1PRDrJdY+WJ02axIsvvsiyZcs4efIkwcHBtimPc+bMYc6c\nOQDk5eXx6KOPkpyczOHDh6+6jfRtQ2IDGPxdf3adKGDN9gw27M5m29ELLEyKZfLwcJyd9AEjIiJy\nI9qsVvadKmLN9gxKqxpwd3Vi8ZQ4Zo2NwtVFrf1FepvrFm6jRo0iMTGRZcuWYTAYePLJJ0lJScHL\ny4tZs2bd8DYilxmNBiYPC2dcQggfH8hh094c3v34HJsP5bF0moUR8ZrSISIici0nMstYtTWdnOJa\nnJ0M3DI2inkTovHyMNk7moh0kRuan/bII490uJ2QkPCl50RGRtrWcLvSNiJf5Gpy4rZJsUwZHs66\nnZlsO3aBF1cfZ2CUL3fOiCc2zNveEUVERHqUrMJqVqWmcyqrAgMwITGERZPjCPR1t3c0EeliurBI\n7M7H05XvzElg5pgoVm1N41h6GU+/fZCbBoeweGocgT76MBIRkb6tuPIia7ZnsO9UezO4IbH+LJ1m\noV+Imv2I9BUq3KTHiAg085M7hnM6q5yVW9PYe6qIg2dLmDUmknkTovFwc7F3RBERkW5VXd/Ehl1Z\nbD2ST2ublegQL+6YbmFwjL+9o4lIN1PhJj3OoBh/nrhvLPtOFrF6ezof7Mthx6cF3DYphukjI9TA\nREREer3GplY+PpDDB/tyaGhqJcjXjcVTLIwdFIxR14GL9Ekq3KRHMhoMTBgSyuiBQXxyMJdNe7P5\n++bzbDmUxx3TLIwaEKQGJiIi0uu0tLax49MC1u/MpKquCU93F759cxzTdOJSpM9T4SY9msnFiXkT\nYpg8PJz3d2aRejSf/11zgvhIH+6aHo8lwsfeEUVERL4xq9XKobMlrN6eQVF5PSYXI7dNjGHO+H5a\n61REABVu4iC8PUzcfcsAZoyOYFVqOkfOl/Lbdw8xNiGYJdMsBKubloiIOKizORW8l5pOxoVqjAYD\n00dGcPukGHw8Xe0dTUR6EBVu4lDCAsz8aMkwzuVWsvJf5zlwppjD50qYOTqS+RNj8HRXAxMREXEM\neSW1rE5N51h6GQBjBgaxeKqFUH8POycTkZ5IhZs4pAFRvvzyO2M4cLqY1dvS+fhALjsvNTC5a/aX\n1xkUERHpKcqrG1i7I5NdJwqwWts/0+6YbsESrun/InJ1KtzEYRkNBsYPDmHUgCC2HMpjw+4sVv4r\njdSjF1g0OZaxCcFqYCIiIj1GXUMzm/Zks/lQHs0tbUQEmVk61cIwS4A+r0TkulS4icNzcTYyZ3w/\nkoaF8f6uLLYeyePldSf5+EAud06PZ0CUr70jiohIH9bc0srmQ3ls3J1NfWMLfl6uLJocx8QhoRiN\nKthE5MaocJNew9PdhW/d3J87Zg3k1TWfcvBMMc/+7TCjBgSxdJquGRARke7V1mZlz8lC1uzIoLy6\nEQ9XZ+6YbmHmqEhMLk72jiciDkaFm/Q6YYFmfrhwCGn5VfzzX2kcPlfCsbRSpl3q0uXlYbJ3RBER\n6cWsVivHM8pY9f/au/P4qKt7/+OvWbJOJiEh+0oIS9jCvsm+C7hCXdoitlKtBay/XtsfSq8/utrb\n1vbWq/Uq1rbWahU1LOvMqZgAACAASURBVAUVVHYIm0DYtwDZ930lmWR+fyREIhgQksxM8n4+HnmQ\nmfnO5J1DkjOf7znfc7amkJFfidnUODNk7tgYLJ5aREtEbo4KN+m0ekX48cyCYXx+Op/3t6bw2ecZ\n7D6WzdyxPZgxIhI3s852iohI20rJKuX9LSmcTi/BAIwbFMq9E3oS4Ovp6Ggi4uJUuEmnZjAYGBEf\nzJDegWw5lMm6nRd4f2sKWw5mMG9SHKP7h2DUBeEiInKLcoqqSNyWwoHT+QAMjuvO/ElxRAb7ODiZ\niHQWKtykSzCbjMwYEcW4gaGsT0rl0wPpvPbvE2zal84DU3sRH+Pv6IgiIuKCistqeHPjabYdzqLB\nbqdnuC/3TY6jb7T6FRFpWyrcpEvx9nTj/im9mDo0gsTt59lzIpff/esQQ3oF8o3JcYQHWhwdUURE\nXMTWw5ms2nyOmtp6Qvy9mD8pjuF9g7S0v4i0CxVu0iUFdvPisbsGMGNkFO9uPsfhcwUcSSlk4pBw\n7h4fi59FC5iIiMhXO5JSyJsfn8Zqcee+Kb2YkBCG2WR0dCwR6cRUuEmXFhvmy7JvDeXwuQLe25LC\n1kOZJB3PYc6YGGaOjMJDyzWLyC2orKxk2bJllJaWUldXx5IlS1i5ciVVVVV4ezduUbJs2TIGDhzo\n4KTydeQUVfHquuOYTEZ+9ugYunnq7ZSItD/9pZEuz2AwMLR3EIN6dmd7chZrd15g9fbzbD2UqQ1S\nReSWrF69mtjYWJ566ilyc3N5+OGHCQoK4je/+Q19+vRxdDy5CdWXbLz4wRGqL9lYNLcfvaP8yc8v\nd3QsEekCNKYv0sRsMjJ1WCT/9f2xzB0bQ0V1HX/98CQ///t+jl8ocnQ8EXFB/v7+lJSUAFBWVoa/\nvxascGUNdjt/WX+C7MKqxgWvBoU5OpKIdCEGu91ud3QIoE3OVgUFWV3mrJcrZQXXyttWWYvKakjc\nfp6kYznYgYE9A7h/Si8ig9puaeeu2K4dxZXydrWsQUHWNkrjGhYtWkRaWhplZWW8+uqr/OEPf8DP\nz4/i4mLi4uJYvnw5np5fvceXzVaPWftOOoV/bTzF25tOk9ArkF88NhaTrmkTkQ6kqZIiXyHA15Pv\n3dGfGSOiWLXlHMfOF3H8wj7GDwrjngk98bd6ODqiiDi5tWvXEh4ezuuvv86pU6dYvnw5P/jBD+jb\nty/R0dGsWLGCt956i0WLFn3laxQXV7VJlq52gqCtHTyTz9ubThPo58miOfEUFVUCzpn1q7hSVnCt\nvMraPlwpK7T/yU0VbiLXERNq5ccPDuHo+UJWbUlhx5Fs9p7M5fZR0dw+OhpPd/0aici1HTx4kPHj\nxwMQHx9PXl4eU6dOxWRqHEGbOnUqH374oSMjyg3ILKjktfUncDcbWTpvEFZvrTwsIh1PY/wiN8Bg\nMJAQF8jPHxnJw7f3xdPdzLpdF3nm1T1sT86iocEpZhyLiJOJiYkhOTkZgMzMTLy9vVm0aBFlZWUA\n7N27l969ezsyolxHVU0dL35whEu19Twytx/RIV1rqq+IOA8NFYh8DSajkUlDIhjdP4SP96bx8b40\n/v7RKT7Zn859U3oxqGeANl4VkWYPPPAAy5cvZ8GCBdhsNn7+859TXFzMd77zHby8vAgJCeGJJ55w\ndEz5Cg0Ndl5Zd5y84mrmjIlhVL8QR0cSkS5MhZvITfB0N3PPhJ5MGhLBmh3n2Xkkmz+9l0z/Hv7c\nP6WXzsiKCAAWi4UXXnjhqvvnzJnjgDTydSVuP8+x80UM7BnAvIk9HR1HRLo4TZUUuQX+Vg++O6cf\nP3tkFANjAzhxsZif/20/r68/QVFZjaPjiYjITdp3MpcP96QS7O/F9+8aoP08RcThNOIm0gaign34\njweGcOxCIas2p7DrWA77TuUxa1QUs0fH4OWhXzUREVeRllvOXz88iYe7iSfmJ2DxdHN0JBERFW4i\nbWlgbHf6fzeAXceyWb39POt3p7LtcBb3jI9lwuBwzNrzR0TEqZVX1fJS4lFq6xpYOm8QEYEWR0cS\nEQE0VVKkzRmNBiYkhPObx8Zy74RYam0NvLnpDCv+uo9DZ/Nxkj3vRUTkS+obGnhl7XEKSmu4a1wP\nhvUJcnQkEZFmGnETaSce7ibuHBfLxCERrN15gW2HM3nxg6P0jerG/VN7ERvm6+iIIiJyhVWbUziZ\nWszQ3oHcNT7W0XFERFrQiJtIO/OzuLNwVl9+sWg0g+O6czq9hF++cYCV646TW1Tl6HgiIgLsOprN\nJwfSCevuzffu6I9RW7uIiJPRiJtIB4kItPDkfYM5mVrMqs3n2HMilwOnP2VCQjhzx8YQ4Ovp6Igi\nIl3Shewy3vj4NF4eZp6Yn6AFpUTEKekvk0gH6xfjz7PfGcHe47msT0ply6FMdhzJYtKQCOaMicHf\n6uHoiCIiXUZpZeNiJPX1jYuRhAZ4OzqSiMg1qXATcQCjwcDYgaHMnRjH2q1n+feui3z2eQbbk7OY\nPCSCOWOi8fNRASci0p5s9Q28vPooxeWXmD+pJwlx3R0dSUTkK6lwE3Egk8nIhIRwxg4IZfexHP69\n6wKfHEhn2+FMpgyLYPboGHwt7o6OKSLSKf3r07OczShlZHwwc8bEODqOiEirVLiJOAGzycjEweHc\nNjCUHUeyWb/7Ihv3pbPlUCbThkdy+6horN4q4ERE2sq2w5lsOZRJZJAPj8zph0GLkYiIk1PhJuJE\nzCYjU4ZGMH5QGNuTs9iQdJGP9qSx+WAm04dHMmtUND5ebo6OKSLi0s5llPLPTWeweJp5Yv4gPNxN\njo4kInJdKtxEnJCb2ci04ZFMSAhj2+EsNuxJZUNSKp99nsHMkVHMHBmFt6cKOBGRr6u4/BJ/Xn2U\nBrudH9wzkKBuXo6OJCJyQ1S4iTgxdzcTM0ZGMXFIOFsPZfLhnlTW7brIJwcymDUyiukjovD21K+x\niMiNqLPV81LiUUora3lwai/69whwdCQRkRumd3wiLsDDzcSsUdFMHhLB5kMZfLQnjTU7GxcymTkq\nmunDI7XvkIhIK+x2O29uPMOF7DLGDghlxsgoR0eSJna7ndLaMjIrcsiqyCazIpv86kLC/YII8Qgl\nxhpFlDUCT7NWW5auTe/0RFyIh7uJ2aNjGgu4gxl8vDeN1dvP88n+dG4fHc3UYRF4uuvXWkTkyzYf\nzGTn0WxiQq08fHtfLUbiIJfqa8mqyCGrMru5UMuqyKHSVtXiOKPByMWytObbBgyEWIKJsUYS4xtF\ntDWSSJ8w3Ey6bEC6Dr3DE3FBXh5m5o7twdRhkXx6IJ2N+9J5f2sKG/elMXt0DFOGReDhpovtRUQA\nTqUW869Pz+Lr7cYT8wbhrr+P7a7B3kBBdeEXo2iVjf8WVBdhx958nAEDQV7d6e0fR4RPKOE+YURY\nwuju5Q/etRy6eIrU8gzSyjJIK88gpzKXvTmfA2AymAj3CSXaGkmMbyQx1ijCLCGYjPr/lc5JhZuI\nC/PyMHPnuFimDY/kkwMZbNqfxqot5/h4XxpzRkczeWiE3qCISJdWUFrNy2uOYTDA4nsHEeDr6ehI\nnU5FbWWLEbTMihyyK3OobahrcZzFzZve3XoS4RNGuE8oET5hhFpC8DBde7ubIJ8ghod4MjxkCNBY\nDOZW5ZNWlkFqeTqpZRlkVGSRXp7Jrqy9ALgZzUT6RDQWck0jc8HegRgNxvZtBJEOoMJNpBPw9nTj\n7vGxTB8RycZ96XxyIJ13Np/jo31pzB0Tw6Qh4biZVcCJSNdyqa5xMZKK6joemtmHPlHdHB3JpdU1\n2MitzCOzIpusypzGfyuyKa0tb3Gc2WAixBJMhE9YY5FmaSzSfN2ttzRF1WgwEmYJIcwSwuiw4QDU\nN9STVZlDalk6aeUZpDYVdRfKUpuf52nyJNoaQXRTMRdjjSTA01/TZcXlqHAT6UQsnm7Mm9iTmSOj\n2LgvjU8PZPD2p2f5aG8ac8fGMCEhHDezzjqKSOdnt9t546NTpOVWMHFwGJOHRjg6ksuw2+2UXCol\ns2mhkMtFWm5VPg32hhbH+nt0Y2D3+MYpjk1FWoh3UIdNVzQZTURZI4iyfvH/W1tfR2ZFVnMRl1aW\nwdmS85wpSWk+xsfN0jzFMrrpujk/D98OySxys1S4iXRCPl5uzJ8Ux4yRUXy8N43NBzP456YzfLgn\nlTtu68H4QWGYTSrgRKTz2rgvnT0ncomL8OXbM7QYyVepsdWQVZnbPHqW2bRwSLWtpsVxHiZ3YqxR\nRDRNcQxvKtK83ZxvHzx3kxuxfjHE+sU031dtqyG9PLPFyNyJotOcKDrdfEw3D78W18tF+0ZicfN2\nxLcgck0q3EQ6MV9vd+6f0otZo6L5aE8qWw5l8o+PT/NhUmMBd9vAUBVwItLpHLtQyHtbz+Hn486S\newdppgGN14dlluVwLC+lqUhrHEUrrClqcZwBA8HeQcQH9CHCEta8YEiAZzeXvk7My+xJH/84+vjH\nNd9XUVvZYnplWlkGRwqOc6TgePMxgZ4BjdfK+UYSY41s2pZA10mKY6hwE+kC/CzuPDitN7ePjubD\npFS2Hs7i7x+dYkPSRe4aF8uYASGYjK7bIYuIXJZXXMWra49jMhpYOm8Q3Xy63t5f5bUVV42gZVfm\nUtdga3Gcj5uFeP/ehDev5hhKqCUE9y6yxL6Pu4X+3fvSv3vf5vtKLpWS2rSCZWpZYzH3eV4yn+cl\nAy23JYhuGpmL9Alz1LcgXcwNFW7PPfccycnJGAwGli9fTkJCQvNjq1at4v3338doNBIfH8+KFSvY\nt28fTz75JL179wagT58+PPvss+3zHYjIDevm48G3ZvRh9pgYNiRdZHtyFq9vOMn63Y0F3Oj+IRiN\nmk4k0tnU1tdSU1eD3W7v1FMGa2ptvJh4lMoaG9+dE09cuJ+jI7Wruvo6sqtym0fPsipyyKzMpry2\nosVxZqOZMEsIPbtH0d0c2Lyqo6+71UHJnVc3Dz+6BfkxOGgA0Hi9X2FNMall6c2jcl/elsBoMBLj\nF0G4d3hTQRdFuLYl6FLqG+qpa6gD2vd36rqF2759+0hNTeXdd98lJSWF5cuX8+677wJQXV3Nhg0b\neOutt3Bzc2PhwoUcOnQIgFGjRvE///M/7RpeRG6Ov9WDBTP7MmdMDOuTUtmRnMVr60+wPukid47r\nwah4FXAibaGyspJly5ZRWlpKXV0dS5YsYcKECQC88847rFy5ks2bN7drhtJL5axI+i/qGuowGox4\nm73wdvPC2+yNl9mz6bb3Ffd74WX2anHb280LD5OHU0+Vs9vtvL7hJJn5lUwbFsmEhHBHR2ozdrud\nopriL63mmENedcFVi4V09/RnUGB/IixNo2g+oQR5BWIymggKspKfX/4VX0WuxWAwEOgVQKBXAMND\nBgON007zqvKbplhmkFaWTkZZFhdK0tlFy20JLk+xjPGN0rYELsZut1Npq6LsUjlltY0f5bUVzZ9f\neX9lXeMG8s9O/iEhxvZbCOm6hVtSUhLTp08HIC4ujtLSUioqKvDx8cHLy4s33ngDaCziKioqCAoK\nIisrq90Ci0jbCfD1ZOGsvswZHc36pIvsPJLDynUnWL87lbvHxzK8bxDGTnx2XqS9rV69mtjYWJ56\n6ilyc3N5+OGH+fjjjyksLOSTTz7pkAwWNy9uCx9FWX0JJZXlVNmqqaqrprC6mHp7/Q2/jgHDFYVe\nU+F3ubBr+vBy87q6AGz6vL3fsK5PSuXz0/n0jerGA9N6tevXak/VtuqrNq3Oqsilpr7lYiGeJk9i\nfaObi7MInzDCLKF46fqrdmc0GAm1hBB6xbYE/t29OXLxHGlXjMxdvS2BB1HWiOb95WJ8o+iubQk6\nXI3t0hfF15XF2BWF2OX7r/c30svsia+7lTBLCAGe/kT4hmGraPUpt+S6hVtBQQEDBgxovh0QEEB+\nfj4+Pj7N961cuZJ//OMfLFy4kKioKLKysjh37hyPP/44paWlLF26lHHjxrX6dfz9vTG3wT5TQUGu\nM+zvSlnBtfIq69fP0K93MA8VVvLuJ2fY/Hk6/7vmGD3CfPnmzL6MGRjWfJwrcaW8yto5+fv7c/p0\n46p1ZWVl+Pv7A/D73/+eH/7wh/zoRz9q9wxmo5n7+9x91WiL3W6nrqGuuZBr/LeKaltN8+dVturm\nx6uv+Dy7Mq9pWtCN8zR5XlHQeX4xynflyF5T8Rdu6E5tpb35mOtNOTt8roA128/T3deDH9w70CUW\nXapvqCevuqDFQiGZFdkUXyppcZzRYCTYO4gIS9/mIi3c0rhYiN7wOw+z0USUNZwoazjjGA1csS1B\neUZjIVeWzrmSC5wtOd/8PIubd3MRd/m6uW4enXuKb3uwNdhajobVllN26Uu3mz5q62tbfS03oxlf\ndyvR1gh83a1YPaz4ul/+8Gn+3Opuvep6UH8vK/kV7Teq/bUXJ7Hb7Vfd99hjj7Fw4UIeffRRhg8f\nTo8ePVi6dCmzZ88mPT2dhQsXsmnTJtzd3b/ydYuLq75ulKu40hQAV8oKrpVXWW+eCfjWtF5MGxbO\nv3ddJOl4Dr95Yz/RwT48NLc/PYMtLvNGwdnatjVdLWtXKvzmzp1LYmIiM2bMoKysjFdffZW9e/fi\n4eHB4MGDb+g12urEJrRt29fV11FZW0VlXTWVtVVU1FY13W78vKq2ioq6pvuaPirqqiioKaTGdulr\nfS0PswcWNy8s7t74uHtjcfPG4t74Yas1sSkpC/dgM9+YPZg67xIuuXvj7e6Fj5s37uavfu/RVlpr\nV7vdTmlNGamlmaSWZJJWmklaSSYZZTnYvrRYSDdPXwaH9iPKL4IYvwhiukUQ7hvapouFuNrvnyvl\nvVbWCAIYxcDm21V11VwsTudcUSopRamcL0rlZNEZThadaT7G38uPuIAe9AqIoad/DHEB0Vg9fK56\n7bbO6mwa7A2U1pRR5VZGSU0ZJdVllF5q/LekpuVHRW1lq69lMBjo5uFLhDWEbl6++Hn60q35w49u\nntbmz73cPG/pvU57tu11C7fg4GAKCgqab+fl5REUFARASUkJZ8+eZeTIkXh6ejJx4kQOHjzI8OHD\nmTNnDgDR0dEEBgaSm5tLVFRUO30bItKWQvy9+d4d/Zk7NoZ/77rI3hO5/Ppv+4gJtXLP+FgS4rq7\nTAEn4khr164lPDyc119/nVOnTvHMM8/g7e3Nyy+/fMOv0RYnNqG9ThAYcceCOxb83YAbrC/qG+qb\nRvaqWoz4VTd9jns9BaWlzSN+l+8vqCwmozQbO186iRwBRuAfJ5PhZMuHzEZzi5G9q67fM3vh5eaN\n95eu9/Mye+Fhcr/u37or27W2vpbsytyrpjpW1LV8U+lmNBNuCW1eJCTC0viv1f1Lb85tUFpUA7Sc\nJnmzXOkkEbhW3q+TNcgQRlD3MMZ2HwNARV1l86InqU0jcwcykzmQmdz8nEDPgMbr5ZpG5m5lWwJH\ntqvdbqemvuZL0xJbjoyVNz1WXld51TWcX2Yxe2P1sBJuCWsxGtb80TRaZnHzvv507UtQeclGJTc/\n17G9T25et3AbN24cL774Ig8++CDHjx8nODi4eZqkzWbj6aefZt26dVgsFo4ePcpdd93FunXryM/P\nZ9GiReTn51NYWEhISMgtfRMi0vHCult47K4BzL2tB5sOZLDzcCYvvH+E2DBf7pkQy8DYABVwIq04\nePAg48ePByA+Pp6UlBRCQ0N59NFHgcaToT/60Y/47//+b0fG7HAmowkfdws+7pZrPt7am58GewM1\nthoq6qr428dHOZtTwNB4P4b09ftiaqetmuq66iuKwirK6yquuZhHa5oXc7ny+r2m4u7y9X7ueUbO\n5qaSWZlNflXhVUVloGcAcX49GjesbroWLciruxapkGY+btfeliCtafGTy5uGH8w7wsG8I0DTtgTe\nQVdcLxdJpE84bg7ayqG2vo7y2msXY+Vfunbsy9tSfJm70Q1fD196eHUnyOqPh92rqQhrWZj5uPvg\nZuxaO5td97sdNmwYAwYM4MEHH8RgMLBixQoSExOxWq3MmDGDJUuWsHDhQsxmM3379mXatGlUVlby\n4x//mM8++4y6ujp+9rOftTpNUkScW0Sghf/70AhmDI9g3c4LHDidz3+vSiYuwpd7xvekfw9dXC1y\nLTExMSQnJzNr1iwyMzMJCwtj48aNzY9PnTq1yxVtt8poMOLt5s3HSTmcPmNnQI9ePD558A3tRWm3\n27lUf6nldX3XuIbv8khgY/HXODJYVFOMrZWFCrzMXsR169E4ima5vFhIiDZrlptyeVuChGtsS3B5\nj7n08kxycvJabEsQbgklpml/uWjfSMItoTe9LUF9Qz0VdZUtVlG86jqypo9qW+ujwkaDsWkRj9Cr\nRsMarxf7oijzNH+x96Irjbp2BIP9WhetOUBb/Ke40n+uK2UF18qrrO3jyqxpueWs3XmBQ2cbp1H3\njvTjngk96Rfj78iILbhq2zo7XeP29VRWVrJ8+XIKCwux2Ww8+eSTjB07tvnxqVOnXnc7gLb62ehM\nP2cHTuXx8ppjBHXz5NmHR+Lj1f6jDNdazKXaVo1/NwsWmx/dPPyc/gSWK/0MgGvldUTWxm0JCpr2\nmGtcACWjIrPFiFbjtgThzZuFx/hG0iMslAvZOc3TFcuvMV2x7FI5FXWVV09LvoIBAxY372sWYi0K\nMg/rTa8s60o/A+AEUyVFRL4sOsTKE/MTSM1pLOAOnyvg9/86RHx0N+4eH0vfaOcp4EQcyWKx8MIL\nL3zl4+29h1tnlJFfwesbTuLhZuKJeQkdUrRB4+IG7iZ33E3uLVb9c7U3ltJ5NG5LEEyoJbh5W4L6\nhnqyKnNJK08ntSyjaXuCDC6Upd3w63qaPPH18CHEEnTNa8Yuj45Z3Xy0yXgHU+EmIjctJtTKD7+R\nwIXsMtbsuMDR84WcevsQ/WL8uXdCT3pFakljEWk7FdV1vPjBES7V1bP4noFEBrftSnsirs505bYE\n4Y3bEtTV15FRkd28v1y9yYan3bPFKJn1iuXu3U26vMlZqXATkVsWG+bLj+4fTEpmKWt2XuD4hSJO\npn7OwNgA7p4QS1y4CjgRuTX1DQ28uvYY+SU13HFbDCPigx0dScQluJnciPWLJtYvGtAosStT4SYi\nbSYuwo+nHhjCmfQS1u68wLELRRy7UERCXHfuHh9LbJivoyOKiIv6YNt5jl8sJiGuO/dM6OnoOCIi\nHU6Fm4i0uT5R3fjJN4dyOq2Y1TsucCSlkCMphQzpFcjd42OJCe06i1OIyK3bczyHj/emERrgzWN3\nDsDo5IuAiIi0BxVuItJu+kb7s+xb3TiVWszqpkVMDp8rYFifIO4eH0uUrk8RketIzSnnbx+dwtPd\nxBPzB+HtqbcuItI16a+fiLQrg8FAvx4BxMf4c+JiMWt2nOfgmXwOnslnRN8g7hofS2SQCjgRuVpZ\nVS0vJR7BZmvgB/MTCOt+7Q27RUS6AhVuItIhDAYDA2ID6N/Dn6Pni1i78zwHTufz+el8RvYL5q5x\nsYQH6k2ZiDSy1TfwyppjFJZd4t4JsQzpHejoSCIiDqXCTUQ6lMFgICGuO4N6BpCcUsjaHRfYdzKP\n/SfzGD0ghLvGxRIa4O3omCLiYO9uPseptBKG9wli7m09HB1HRMThVLiJiEMYDAaG9ApkcFx3Dp8t\nYM3OC+w5nsveE7mMHRDKneN6EOKvAk6kK/p0XyqffZ5BRKCFR+b202IkIiKocBMRBzMYDAztE8Tg\n3oEcPJ3P2l0X2H0shz3Hc7ltUCh33taDoG5ejo4pIh0kJauUP79/BG8PM0/MH4SXh96qiIiACjcR\ncRJGg4ER8cEM6xvEgVN5rN15gZ1Hskk6lsO4QWHccVsMgX4q4EQ6s5KKS/w58SgNDQ08fvcggjXq\nLiLSTIWbiDgVo8HAqH4hjOgbzL5TuazdeZHtyVnsOprNhMHh3DE2hgBfT0fHFJE2Vmdr4M+rj1JS\nUct37+jPwJ7dHR1JRMSpqHATEadkNBoY0z+UkfHB7D2Ry7pdF9l6KJOdR7KYNDiCOWNj8Ld6ODqm\niLSRtz89Q0pmGaP7h3Dv5F4UFFQ4OpKIiFNR4SYiTs1kNHLbwDBG9w8h6Vgu63Zd4LODGWxLzmLy\n0HDmjonBz0cFnIgr23Iok22Hs4gO9uE7s+MxaDESEZGrqHATEZdgMhoZnxDGmAEh7D6Ww793XeTT\nAxlsP5zF5KERzBkTg6/F3dExReRrOpNewtufnMHHy42l8wfh4WZydCQREaekwk1EXIrZZGTi4HBu\nGxjKziPZrE+6yKb96Ww9nMm0YZHcPjoaq7cKOBFXUFRWw8urj2K3w+J7BmoBIhGRVqhwExGXZDYZ\nmTw0gnGDwthxJIv1uy/y0d40Nh/KZPrwSBbM6e/oiCLSitq6el5KPEpZVR3fmt6b+Bh/R0cSEXFq\nKtxExKW5mY1MHRbJhIQwth7O4sOkVDYkpbLtcBZzxsQwbXgEbmZNvRJxJna7nX9sPM3FnHLGDQpl\n2vBIR0cSEXF6RkcHEBFpC25mEzNGRPHbx8dy35Q47MCqLedYvnIPu45m09Bgd3REEWny6YEMdh/L\nITbMl4Wz+moxEhGRG6DCTUQ6FXc3E7NHx/Da8uncPiqa0so6Xt9wkp/9bT9Hzxdit6uAE3GkExeL\neHfzOXwt7iydN0gj4iIiN0iFm4h0SlZvd+6f2ovnHhvNbQNDycyv4L9XJfP8O4e5mFPm6HgiXVJ+\nSTWvrD2OwQBL7h2ovRhFRL4GFW4i0qkF+nnxvTv6s+K7IxkYG8DJ1GJ+8fcDvLruOPkl1Y6OJ9Jl\nXKptXIykorqOb8/sQ+/Ibo6OJCLiUrQ4iYh0CdEhVv7jgSGcuFjEe1tS2HsilwOn8pgyLII7b+uh\nLQRE2pHdbudvH50kPa+CyUMjmDwkwtGRRERcjgo3EelS+vcI4Nnv+LPvZC6J287z6YEMdh3NZvbo\nGGaMjNLmvyLtLvBR3wAAGCBJREFU4KO9aew7mUevSD++Nb23o+OIiLgkFW4i0uUYDQbG9A9leJ9g\nth7K5N+7L5K4/TybD2Zwz4SejBsUismomeRy6yorK1m2bBmlpaXU1dWxZMkSamtrWblyJW5ubgQE\nBPD73/8eD4/Oe63X0fOFfLA1BX+rB0vuGYjZpN8tEZGbocJNRLosN7ORGSOjGDcojI/2pvLJ/nT+\n/tEpNu1PZ/6kngzpFahlyuWWrF69mtjYWJ566ilyc3N5+OGHCQkJ4S9/+QtWq5VnnnmGTZs2ceed\ndzo6arvILari1bXHMZmMLJ03CD+fzlugioi0N532EpEuz9vTzPxJcfzm+2OZODiM7MJKXvzgKL99\n6yApmaWOjicuzN/fn5KSEgDKysrw9/fnjTfewGq1YrPZyM/PJyQkxMEp20f1JRsvJh6l6pKNh2/v\nS2yYr6MjiYi4NBVuIiJN/K0efGd2P36xaDRDegVyJqOUX7/5OX9efZScoipHxxMXNHfuXLKyspgx\nYwYLFixg2bJlACQmJjJ9+nSio6MZNWqUg1O2vQa7nb+sP0FWQSXTR0QyblCYoyOJiLg8g91JdqPN\nzy+/5dcICrK2yet0BFfKCq6VV1nbhytlhbbJeya9hPe2nCMlqwyjwcCkIeHcNT4WP0vbrkDpSm3b\nFlmDgqxtlMb5rV27lgMHDvDLX/6SU6dOsXz5chITEwGw2WwsW7aMyZMntzpV0marx+xim1T/a9Np\n3t54ioRegfzisbGYdF2biMgt0zVuIiJfoU9UN5Y/NJzPT+fzwbYUthzKZPexHGaNimLWqGi8PPQn\nVFp38OBBxo8fD0B8fDwZGRls3bqVyZMnYzabmTZtGvv27Wu1cCsubpvR3o46QXDoTD5vbzxFd19P\nFs2Jp6io8mu/Rlc7mdFRXCkruFZeZW0frpQV2v/kpk6BiYi0wmAwMCI+mF9+bzQPzeyDh7uJdbsu\n8syrSWw+mIGtvsHREcWJxcTEkJycDEBmZiZWq5UVK1aQm5sLwJEjR4iNjXVkxDaVWVDJyvUncDcb\neWL+IO2PKCLShnS6WETkBphNRqYMi2TswFA27kvn471p/HPTGT7Zn878SXEM7xukFSjlKg888ADL\nly9nwYIF2Gw2fvWrX1FbW8uSJUtwd3cnMDCQJ5980tEx20RVTR0vfXCES7X1PH73AKJDus6UWBGR\njqDCTUTka/B0N3P3+FgmD41g3a4LbD+cxctrjtEz3Jf7JsfRN9rf0RHFiVgsFl544YWr7p80aZID\n0rSfhgY7r647QW5xNbPHRDOqX+dcKVNExJE0VVJE5Cb4Wdx5aGZffvW90YzoG8T5rDJ++/YhXngv\nmcz8CkfHE+lQq3ec5+j5Qgb2DGD+xDhHxxER6ZQ04iYicgtCArxZfO8gUrJKeW9LCskphRw5X8j4\nQWHcM6En/lZtOCyd276TuWxISiXY34vv3zUAo1FThkVE2oMKNxGRNhAX7seybw0lOaWQD7amsONI\nNntP5DJjZBSzR8fg7ak/t9L5pOWW89cPT+LhbuKJeYOweLo5OpKISKeldxIiIm3EYDAwpFcgCT27\ns+toNmt2XmBDUirbDmdxx209mDI0AjezZqhL51BRXcdLiUeprWtgyb2DiAjycXQkEZFOTe8gRETa\nmNFoYMLgcJ57bAzzJ/WkvqGBdz47y09f28Oe4zk02O2OjihyS+obGvjfNccoKK3hrnE9GN43yNGR\nREQ6PRVuIiLtxMPNxNyxPfiv749lxogoissvsfLfJ/jl3w9w/GKRo+OJ3LT3tqRwMrWYIb0CuWt8\n59mHTkTEmWmqpIhIO7N6u/PN6b2ZPiKS1dvPs+dELn945zADYgO4b3IcQUHa70pcx+5j2Wzan05Y\nd28evbM/Ru1fKCLSIVS4iYh0kKBuXjx21wBmjYrmva3nOH6hiBMXipg8PJLZo6II9PNydESRVl3I\nLuPvH53Gy8PME/MT8PLQ2wgRkY6iv7giIh0sJtTKjx8cyrELhby3JYUtn2ew43Am04ZHMndsD3y8\ntDKfOJ/SylpeSjxKfX0DS+cNJDTA29GRRES6FBVuIiIOMjC2O/17BHAivZQ31p9g4750diRnM3ds\nDNOGR+LuZnJ0RBEAbPUN/O/qoxSXX2L+pJ4kxAU6OpKISJejxUlERBzIaDAwZXgUzz02mvun9MJg\ngPe2pvDMyj3sPJJNQ4NWoBTH+9dnZzmTUcqI+GDmjIlxdBwRkQ61detnN3Tcr3/9a7KyMtsthwo3\nEREn4GY2cfvoaP7r8bHMHh1NeVUdf/3wJD/72z6OpBRg1xYC4iDbk7PYcjCTyCALi+b0w6DFSESk\nC8nOzuLTTzfe0LE//elPCQ+PaLcsmiopIuJELJ5u3DelF9OGR7J6x3l2H83hT+8dIT66G/dN6UVs\nmK+jI0oXci6zlDc3nsbiaWbp/AQ83DV9V0S6lj/+8becPHmcCRNGMnPmbLKzs/jTn17mN7/5Bfn5\neVRXV/PII48xbtwEHnroIZYu/Q+2bPmMysoK0tJSyczM4Ic/fIqxY8fdchYVbiIiTijA15NFc/sz\na2Q0729L4UhKIb984wAj44OZP6knwf5aGELaV3H5Jf6ceJQGu53H7xlIcDeteioijrNq8zn2n8pr\n09ccGR/M/VN7tXrMN7/5EImJq4iNjSMt7SIvv/wXiouLGDVqDLNn30FmZgbPPvs048ZNaPG8vLxc\nnn/+f9izZzdr136gwk1EpLOLDPbh/9w3mFOpxby3tbHTOngmn8lDIrhzXA98Le6OjiidUJ2tgT+v\nPkppZS0PTu3FgB4Bjo4kIuJw/foNAMBq9eXkyeOsW5eIwWCkrKz0qmMTEoYAEBwcTEVFRZt8/Rsq\n3J577jmSk5MxGAwsX76chISE5sdWrVrF+++/j9FoJD4+nhUrVmAwGFp9joiIfD3xMf7858IR7D+V\nR+K283x2MINdx7KZPTqamSOjNYVN2ozdbufNTac5n1XG2AEhzBgZ5ehIIiLcP7XXdUfH2pubW+N2\nPZ988jFlZWX8+c9/oaysjO9976GrjjWZvuiX2+o69esWbvv27SM1NZV3332XlJQUli9fzrvvvgtA\ndXU1GzZs4K233sLNzY2FCxdy6NAhbDbbVz5HRERujsFgYFS/EIb1CWLb4SzW7brA6h0X2Hwok7vH\nxzIhIQyTUWtOya3ZfDCTnUeyiQm18vDt8VqMRES6NKPRSH19fYv7SkpKCAsLx2g0sm3bZurq6jom\ny/UOSEpKYvr06QDExcVRWlraPNzn5eXFG2+8gZubG9XV1VRUVBAUFNTqc0RE5NaYTUamDY/kv74/\nljtv60H1JRv/+Pg0/+/1fRw8k68VKOWmnU4r5p3PzmL1duOJeYO0l6CIdHkxMbGcPn2KysovapnJ\nk6eye/cOnnzyB3h5eREcHMzf/vZau2e57ohbQUEBAwYMaL4dEBBAfn4+Pj4+zfetXLmSf/zjHyxc\nuJCoqKgbes6X+ft7YzbfegcRFGS95dfoKK6UFVwrr7K2D1fKCq6V92azPhbpzzdm9OXtjaf4ZF8a\nLyUepV+PAL57xwD6xbbPdUmu1K5y4wpLa3h5zTEAFt8zkABfTwcnEhFxPH9/fxITN7S4LywsnDfe\neKf59syZs4HG/jE/v5yePb+Y0tmzZy9eemllm2T52ouTXOtM7mOPPcbChQt59NFHGT58+A0958uK\ni6u+bpSrXG4sV+BKWcG18ipr+3ClrOBaedsi6wOT45g4KJT3t6Zw6GwB//elHQztHcg3JscR1t3S\nRknbJqsKP+dzqa6eFxOPUF5Vx4KZfegb7e/oSCIi8iXXLdyCg4MpKChovp2Xl0dQUBDQOL/z7Nmz\njBw5Ek9PTyZOnMjBgwdbfY6IiLSPsO4WnpifwNmMEt7b0ljAJZ8rZMLgMO4eH0s3Hw9HRxQnZLfb\neePjU6TlVjBxcBhThrbf5rEiInLzrnuN27hx49i4sXG38OPHjxMcHNw85dFms/H0009TWVkJwNGj\nR4mNjW31OSIi0r56R3bjmQXDWDpvEMH+Xmw7nMXTryaxevt5qi/ZHB1PnMym/ensOZ5LXLgv357R\nV4uRiIg4qeuOuA0bNowBAwbw4IMPYjAYWLFiBYmJiVitVmbMmMGSJUtYuHAhZrOZvn37Mm3aNAwG\nw1XPERGRjmMwGBjWJ4jBvbqz40g2a3dc4N+7L7L1cCZ3jYtl0pBwzCatQNnVHb9QxKot5/DzcWfx\nvYNwM+tnQkTEWd3QNW4//vGPW9yOj49v/nzevHnMmzfvus8REZGOZzIamTwkgrH9Q9m0P40P96bx\n1idn+GR/OvMm9WRkfLBGWLqovJJqXll7DJPRwNJ7B+Fv1VRaERFn9rUXJxEREdfj4W7iznGxTBoS\n0TjydiiTV9YeZ+O+NO6b3Iv4GC1G0R4qKytZtmwZpaWl1NXVsWTJEoKCgvjFL36B0WjE19eXP/zh\nD3h5eXVorppaGy99cITKGhvfnR1PXIRfh359ERH5+jQnQkSkC/G1uPPtGX349aOjGdUvmAvZ5fzu\nX4f403vJZORpv822tnr1amJjY3nzzTd54YUX+PWvf82vfvUrnn76af75z38SExNDYmJih2ay2+38\ndcNJMvIrmTosggmDwzv064uIdDbf+MadVFXd+gr516MRNxGRLijY35vH7x7IrFFlvLflHEdSCjma\nUshtg0K5d0JP7eHVRvz9/Tl9+jQAZWVl+Pv788orrzQv2BUQEEBJSUmHZtqQlMqB0/n0ierGg9N6\nd+jXFhGRm6fCTUSkC4sN8+Un3xzK0fOFvLc1hV1Hc9h3Mo/pwyOZOzYGb083R0d0aXPnziUxMZEZ\nM2ZQVlbGq6++2ly0VVVVsXbtWl544YUOy5N8roDV288T4OvB4nsGaoEaEZFWPPLIt3nuuT8QGhpK\nTk42zzzzFEFBwVRXV1NTU8OPfvQT+vcf2GF5VLiJiHRxBoOBhLhABsZ2Z/exHFbvOM9He9PYnpzF\n3LE9mDY8AjezydExXdLatWsJDw/n9ddf59SpUyxfvpzExESqqqr4wQ9+wCOPPEJcXFyrr+Hv7425\nDdo/I6+c19afwM1s5NlHxhAX1e2WX7M9udJG7craflwpr7K2j8tZ3zz8AXvSD7bpa4+JGsZDQ+Z/\n5eO33z6LI0f2MWjQt/noo9Xcfvss4uPjmT59OklJSbz99tu8+OKLmExGAgN9WuRtDyrcREQEAKPR\nwPiEMEb1C+azzzNYn5TKqi3n+OzzDOZN7MnoASGOjuhyDh48yPjx44HGFZnz8vKora1l8eLF3HHH\nHddclfnLiotv/bqJqhobv3nrIFU1Nh69sz9+niby88tv+XXbS1CQ1anzXUlZ248r5VXW9nFl1qrq\nWuob7G36+lXVta22xYgR43jppT8xc+ZdfPzxJpYu/RHvvPMmr7yykrq6Ojw9PcnPL6e+voGCggos\nFsstt21rhZ8KNxERacHdzcTsMTFMGBzOhqSLfPZ5Bq+tP8HGfWk88eBQuntr+uSNiomJITk5mVmz\nZpGZmYnFYuH1119n1KhR3HfffR2SwW6385f1J8jMr2DWqCjGDgjtkK8rItKW5vW6g3m97ujQr9mz\nZxyFhfnk5uZQXl7Ojh1bCQwM5tlnf8mpUyd46aU/dWgeFW4iInJNPl5uPDC1N9OGR7J6+wX2HM/h\nr+uO85MHhzg6mst44IEHWL58OQsWLMBms/Gzn/2Mn/zkJ0RGRpKUlATA6NGjWbp0abtlKC6/xOFz\nBQzpHcQ3Jrc+LVNERFoaO3Y8K1e+zIQJkygpKSYurnFRp23btmCz2To0iwo3ERFpVaCfF4/e2Z+7\nxvUgJNgXOrijcmUWi+WqxUd27tzZoRkCfD35z4UjGNo/lNKS9l+uWkSkM5k0aQqPP/4If//7v6ip\nqeZXv1rBli2fMn/+/Xz66SY2bFjXYVlUuImIyA0JCfAmyN/LZa6NkC/0DPfF3U0LzIiIfF39+g1g\n27a9zbffeuv95s/Hj58EwNy5d3VIFq0DLCIiIiIi4uRUuImIiIiIiDg5FW4iIiIiIiJOToWbiIiI\niIiIk1PhJiIiIiIi4uRUuImIiIiIiDg5FW4iIiIiIiJOToWbiIiIiIiIk1PhJiIiIiIi4uRUuImI\niIiIiDg5g91utzs6hIiIiIiIiHw1jbiJiIiIiIg4ORVuIiIiIiIiTk6Fm4iIiIiIiJNT4SYiIiIi\nIuLkVLiJiIiIiIg4ORVuIiIiIiIiTs7s6AA347nnniM5ORmDwcDy5ctJSEhofmz37t388Y9/xGQy\nMXHiRJYsWeLApI1ayzt16lRCQ0MxmUwAPP/884SEhDgqKmfOnGHx4sV85zvfYcGCBS0ec8a2bS2v\ns7Xt7373Oz7//HNsNhvf//73mTlzZvNjzta2rWV1pnatrq7m6aefprCwkEuXLrF48WKmTJnS/Lgz\ntev1sjpTu15WU1PDHXfcweLFi5k3b17z/c7UrnI1V+ojXal/BNfqI9U/tg/1j+1DfeQNsruYvXv3\n2h977DG73W63nzt3zn7//fe3eHz27Nn2rKwse319vf2b3/ym/ezZs46I2ex6eadMmWKvqKhwRLSr\nVFZW2hcsWGD/z//8T/ubb7551ePO1rbXy+tMbZuUlGT/3ve+Z7fb7faioiL7pEmTWjzuTG17vazO\n1K4bNmywr1y50m632+0ZGRn2mTNntnjcmdr1elmdqV0v++Mf/2ifN2+e/YMPPmhxvzO1q7TkSn2k\nK/WPdrtr9ZHqH9uH+sf2oz7yxrjcVMmkpCSmT58OQFxcHKWlpVRUVACQnp6On58fYWFhGI1GJk2a\nRFJSkiPjtprX2bi7u/Paa68RHBx81WPO2Lat5XU2I0eO5IUXXgDA19eX6upq6uvrAedr29ayOps5\nc+bw6KOPApCdnd3i7JuztWtrWZ1RSkoK586dY/LkyS3ud7Z2lZZcqY90pf4RXKuPVP/Y8VmdjSv1\nj6A+8ka53FTJgoICBgwY0Hw7ICCA/Px8fHx8yM/PJyAgoMVj6enpjojZrLW8l61YsYLMzEyGDx/O\nU089hcFgcERUzGYzZvO1fyScsW1by3uZs7StyWTC29sbgPfff5+JEyc2D/c7W9u2lvUyZ2nXyx58\n8EFycnJ45ZVXmu9ztna97FpZL3Omdv3tb3/Ls88+y5o1a1rc76ztKo1cqY90pf4RXKuPVP/YPtQ/\ntj/1ka1zucLty+x2u6MjfC1fzvvDH/6QCRMm4Ofnx5IlS9i4cSO33367g9J1Ls7Ytp9++invv/8+\nf/3rXx2a40Z8VVZnbNd33nmHkydP8pOf/IR169Y5vKNszVdldaZ2XbNmDUOGDCEqKsohX1/ajiv1\nkeofO44ztq36x/bhSv0jqI+8HpebKhkcHExBQUHz7by8PIKCgq75WG5ursOnCbSWF+Cee+6he/fu\nmM1mJk6cyJkzZxwR87qcsW2vx9nadseOHbzyyiu89tprWK3W5vudsW2/Kis4V7seO3aM7OxsAPr1\n60d9fT1FRUWA87Vra1nBudp169atfPbZZ9x///289957vPzyy+zevRtwvnaVllypj+ws/SM4X9te\nj7O1rfrHtudK/SOoj7xRLle4jRs3jo0bNwJw/PhxgoODm6dVREZGUlFRQUZGBjabjS1btjBu3DhH\nxm01b3l5OYsWLaK2thaA/fv307t3b4dlbY0ztm1rnK1ty8vL+d3vfserr75Kt27dWjzmbG3bWlZn\na9cDBw40n/EsKCigqqoKf39/wPnatbWsztauf/rTn/jggw9YtWoV9913H4sXL+a2224DnK9dpSVX\n6iM7S/8Izte2rXG2tlX/2D5cqX8E9ZE3ymB3pXkUTZ5//nkOHDiAwWBgxYoVnDhxAqvVyowZM9i/\nfz/PP/88ADNnzmTRokUOTtt63jfeeIM1a9bg4eFB//79efbZZx02jH3s2DF++9vfkpmZidlsJiQk\nhKlTpxIZGemUbXu9vM7Utu+++y4vvvgisbGxzfeNHj2avn37Ol3bXi+rM7VrTU0NP/3pT8nOzqam\npoalS5dSUlLilH8PrpfVmdr1Si+++CIREREATtmucjVX6iNdpX8E1+oj1T+2D/WPjsvrTG17pY7u\nI12ycBMREREREelKXG6qpIiIiIiISFejwk1ERERERMTJqXATERERERFxcircREREREREnJwKNxER\nERERESenwk1ERERERMTJqXATERERERFxcircREREREREnNz/B1rc2bCXdw5NAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f709d49e6a0>"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot performance\n",
    "trainer.plot_performance()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 51
    },
    "colab_type": "code",
    "id": "wYGUHK0GsfhP",
    "outputId": "95fd16d0-5642-4e4c-a617-f4cb41cfcb58"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test loss: 0.46\n",
      "Test Accuracy: 84.0%\n"
     ]
    }
   ],
   "source": [
    "# Test performance\n",
    "trainer.run_test_loop()\n",
    "print(\"Test loss: {0:.2f}\".format(trainer.train_state['test_loss']))\n",
    "print(\"Test Accuracy: {0:.1f}%\".format(trainer.train_state['test_acc']))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "GO8v_uUisfd9"
   },
   "outputs": [],
   "source": [
    "# Save all results\n",
    "trainer.save_train_state()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "VbQdvtK3twoW"
   },
   "source": [
    "## Freeze embeddings"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "XrMm4K61t21h"
   },
   "source": [
    "Now we're going to freeze our GloVe embeddings and train on the supervised task. The only modification in the model is to turn on `freeze_embeddings`:\n",
    "\n",
    "```python\n",
    "if freeze_embeddings:\n",
    "    self.embeddings.weight.requires_grad = False\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "ilV_QbtktzH3"
   },
   "outputs": [],
   "source": [
    "args.freeze_embeddings = True"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 204
    },
    "colab_type": "code",
    "id": "7NdD-iP6tzFQ",
    "outputId": "78702aec-ca66-422f-c412-71f0003cf22f"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<bound method Module.named_modules of NewsModel(\n",
      "  (embeddings): Embedding(3407, 100, padding_idx=0)\n",
      "  (conv): ModuleList(\n",
      "    (0): Conv1d(100, 100, kernel_size=(2,), stride=(1,))\n",
      "    (1): Conv1d(100, 100, kernel_size=(3,), stride=(1,))\n",
      "    (2): Conv1d(100, 100, kernel_size=(4,), stride=(1,))\n",
      "  )\n",
      "  (dropout): Dropout(p=0.1)\n",
      "  (fc1): Linear(in_features=300, out_features=100, bias=True)\n",
      "  (fc2): Linear(in_features=100, out_features=4, bias=True)\n",
      ")>\n"
     ]
    }
   ],
   "source": [
    "# Initialize model\n",
    "model = NewsModel(embedding_dim=args.embedding_dim, \n",
    "                  num_embeddings=len(vectorizer.title_vocab), \n",
    "                  num_input_channels=args.embedding_dim, \n",
    "                  num_channels=args.num_filters, hidden_dim=args.hidden_dim, \n",
    "                  num_classes=len(vectorizer.category_vocab), \n",
    "                  dropout_p=args.dropout_p, pretrained_embeddings=embeddings,\n",
    "                  freeze_embeddings=args.freeze_embeddings,\n",
    "                  padding_idx=vectorizer.title_vocab.mask_index)\n",
    "print (model.named_modules)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 102
    },
    "colab_type": "code",
    "id": "6NJhm2EOtzCV",
    "outputId": "047c1aa8-a44a-4010-c824-a89e30f4157c"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[EPOCH]: 0 | [LR]: 0.001 | [TRAIN LOSS]: 0.53 | [TRAIN ACC]: 80.7% | [VAL LOSS]: 0.48 | [VAL ACC]: 82.7%\n",
      "[EPOCH]: 1 | [LR]: 0.001 | [TRAIN LOSS]: 0.45 | [TRAIN ACC]: 83.6% | [VAL LOSS]: 0.46 | [VAL ACC]: 83.4%\n",
      "[EPOCH]: 2 | [LR]: 0.001 | [TRAIN LOSS]: 0.42 | [TRAIN ACC]: 85.0% | [VAL LOSS]: 0.46 | [VAL ACC]: 83.8%\n",
      "[EPOCH]: 3 | [LR]: 0.001 | [TRAIN LOSS]: 0.39 | [TRAIN ACC]: 85.9% | [VAL LOSS]: 0.46 | [VAL ACC]: 83.6%\n",
      "[EPOCH]: 4 | [LR]: 0.001 | [TRAIN LOSS]: 0.36 | [TRAIN ACC]: 87.0% | [VAL LOSS]: 0.46 | [VAL ACC]: 83.7%\n"
     ]
    }
   ],
   "source": [
    "# Train\n",
    "trainer = Trainer(dataset=dataset, model=model, \n",
    "                  model_state_file=args.model_state_file, \n",
    "                  save_dir=args.save_dir, device=args.device,\n",
    "                  shuffle=args.shuffle, num_epochs=args.num_epochs, \n",
    "                  batch_size=args.batch_size, learning_rate=args.learning_rate, \n",
    "                  early_stopping_criteria=args.early_stopping_criteria)\n",
    "trainer.run_train_loop()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 335
    },
    "colab_type": "code",
    "id": "oojDHLowty_9",
    "outputId": "72dae3b4-af2b-4b53-e5dc-9f6cc92cebac"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA3MAAAE+CAYAAAAnEEnhAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3XlcVXXi//HXvRcuCFwQ9LKqbIoL\n7lvmlmIKpqa2aZnZNvOtrH7T1HydYSprlqaZlvnWVDNtY4vN5Ixhtpj7viGJoqLgvoDIosimst7f\nHyiCuwnce+H9fHzn273nnnvO+94g7vuecz4fg81msyEiIiIiIiJOxWjvACIiIiIiInL9VOZERERE\nRESckMqciIiIiIiIE1KZExERERERcUIqcyIiIiIiIk5IZU5ERERERMQJqcyJNIKOHTty7Ngxe8cQ\nERFpNJMnT+b222+3dwyRJk1lTkRERETq1e7du7FYLAQHB7NlyxZ7xxFpslTmROyotLSUF198kdjY\nWEaPHs2rr75KZWUlALNnz2b06NHExcVx1113sWfPnisuFxERcRTz5s0jLi6OsWPH8vXXX9cs//rr\nr4mNjSU2NpZf/epXlJWVXXZ5YmIiI0eOrHlu7ft/+9vfeP7557nrrrv45JNPqKqq4uWXXyY2NpaY\nmBh+9atfUV5eDsCJEyd47LHHGDFiBOPGjWPt2rWsXLmSsWPH1sl8xx13sHTp0oZ+a0TqlYu9A4g0\nZ59++inHjh3j+++/p6Kigvvvv5/vvvuOESNG8NZbb7FixQq8vLz44YcfWLlyJUFBQZdc3qFDB3u/\nFBEREQAqKytZsmQJ06dPx2Qy8cYbb1BWVkZOTg5//vOf+frrr/H39+epp57is88+Iy4u7pLLu3Xr\ndsX9rFq1ivnz5+Pn58eiRYv48ccf+e6776iqqmLixIksWLCA8ePH88YbbxAZGck//vEPdu7cyUMP\nPcSaNWvIzc0lLS2NTp06cfToUQ4fPszQoUMb6V0SqR8qcyJ2tHLlSh5++GFcXFxwcXFh3LhxrFu3\njttuuw2DwcDcuXMZO3Yso0ePBqC8vPySy0VERBzF2rVr6datG15eXgD079+fFStWcPLkSXr16kVA\nQAAAb7zxBiaTia+++uqSyzdv3nzF/fTo0QM/Pz8AYmNjGT58OK6urgB069aNI0eOANWl78MPPwSg\nS5cuLFu2DLPZTGxsLN9//z2dOnVi6dKljBgxArPZXP9viEgD0mmWInZ04sQJfHx8au77+Phw/Phx\nXF1d+eSTT0hOTiY2Npb77ruP9PT0yy4XERFxFAkJCaxcuZK+ffvSt29fFi9ezLx588jPz8fb27tm\nPTc3N1xcXC67/Gpq//08ceIEM2bMIDY2lri4OJYtW4bNZgPg5MmTWCyWmnXPlcwxY8bw/fffA7B0\n6VJuu+22G3vhInagMidiR61bt+bkyZM190+ePEnr1q2B6m8P3377bTZs2MDgwYOZOXPmFZeLiIjY\nW0FBAZs2bSIxMZEff/yRH3/8kaSkJLZv347RaCQ/P79m3eLiYvLy8vD19b3kcpPJVHMdOUBhYeFl\n9/vXv/4VFxcXvv32WxYuXMgtt9xS81jLli3rbD8jI4Py8nL69etHRUUFK1asYM+ePQwcOLC+3gaR\nRqMyJ2JHw4YNY+7cuVRWVnLq1Cnmz5/PLbfcQnp6Ok8//TRlZWWYzWa6du2KwWC47HIRERFH8P33\n3zNgwIA6pyu6uLgwePBgysrKSE5OJiMjA5vNxsyZM5k7dy633HLLJZdbrVZyc3M5fvw4lZWVfPvt\nt5fd7/Hjx4mKisJsNpOWlsaWLVs4deoUADExMcybNw+AvXv3cscdd1BZWYnRaOS2227j97//PTEx\nMTWnaIo4E10zJ9JIpk6dislkqrn/hz/8galTp3LkyBHGjBmDwWAgLi6u5jq4Nm3aMHbsWFxdXfH0\n9OTFF18kKirqkstFREQcwddff820adMuWj5y5Ejee+89fve73zFt2jRMJhPdunXjoYcews3N7bLL\n77zzTiZMmEBwcDDjx49n165dl9zvww8/zIwZM0hISKBv377MmDGD3/72t3Tv3p1f/epXzJgxg5iY\nGDw9PXn99ddxd3cHqk+1nDVrlk6xFKdlsJ07oVhEREREpBnJy8tj4sSJrFy5ss4XriLOQqdZioiI\niEiz9Pbbb3PvvfeqyInTUpkTERERkWYlLy+PESNGkJeXx8MPP2zvOCI/mU6zFBERERERcUI6Mici\nIiIiIuKEVOZERERERESckENPTZCbW1Qv2/H19SA//1S9bKuhKWvDcKas4Fx5lbVhOFNWqJ+8Vqul\nntI0D/XxN7I5/pw1FmVtGMracJwpb3PLeqW/j83iyJyLi/OMUKSsDcOZsoJz5VXWhuFMWcH58ko1\nZ/v35kx5lbVhKGvDcaa8ynpesyhzIiIiIiIiTY3KnIiIiIiIiBNSmRMREREREXFCKnMiIiIiIiJO\nSGVORERERETECanMiYiIiIiIOCGHnmdORESkqSkpKWHGjBkUFBRQXl7O9OnT+eCDD2oez8nJYeLE\niTz22GN2TCkiIs5AZU5ERC5r5cplDBs24qrr/fGPf2Ts2DsJDg5phFTObd68eYSHh/Pss8+SnZ3N\ntGnTWLhwYc3jjz76KOPHj7djQhERcRY6zVJERC4pK+soS5cuuqZ1f/vb36rIXSNfX19OnjwJQGFh\nIb6+vjWPrV+/nrCwMIKCguwVT0REnEiTPzK3ZttRBnQPwdXeQUREnMybb/6ZXbtSGTKkH6NGjSYr\n6yj/93/v8ac//Y7c3BxOnz7Nww//nEGDhjB16lSefPKXrFixjJKSYg4fPkRmZgZPP/0sN988yN4v\nxaGMGTOGhIQERo4cSWFhIe+//37NY5999hnx8fF2TCciIvXBZrOxff9x3I8WERVsabD9NOkyV1pW\nySc/pPHt+kO88EAfLB5me0cSEXEa9947lYSE/xAeHsnhwwd5772PyM8/Qf/+Axg9eiyZmRm88MKv\nGTRoSJ3n5eRk8/rrb7Nx43rmz/9KZe4C8+fPJzg4mI8//pi0tDTi4+NJSEggOzubU6dO0a5du2va\njq+vBy4uphvOY7U23IeMhuBMeZW1YShrw3GmvI6a1WazsXHHMb5cks7+zAJ8vMzMfnl0g+2vSZc5\nN7OJCUMimLd6Px98k8oz9/TEaDTYO5aIyHX7z/K9JKXl1Os2+3Xy556Y9te0bufO0QBYLN7s2pXK\nN98kYDAYKSwsuGjd7t17AuDv709xcXH9BW4ikpOTGTx4MACdOnUiJyeHyspKVq1axYABA655O/n5\np244i9VqITe36Ia301icKa+yNgxlbTjOlNcRs1bZbCSn5/Lt+oMcySnGAPTv7M+D47recNYrFdcm\nXeYAxtwcSkZeCUk7s5m/9gATh0bYO5KIiNNxda0+WX3JkoUUFhby7rsfUVhYyKOPTr1oXZPp/NEi\nm83WaBmdRWhoKCkpKcTGxpKZmYmnpycmk4nt27czfPhwe8cTEZHrUGWz8WNaDt+uP0hmbgkGAwyI\nDmDszWEEt/Zs8OLZ5Muc0WDgl/f25qnXV/Dt+oNEBHvTo31re8cSEbku98S0v+ajaPXFaDRSWVlZ\nZ9nJkycJCgrGaDSyatVyysvLGzVTUzBp0iTi4+O5//77qaio4KWXXgIgNzeXVq1a2TeciIhck6oq\nG5vSsvlu/SGO5lWXuIFdAxlzcyhBrTwbLUeTL3MAXh5mpk/sxh8/38xH3+3kxQf7YW3Zwt6xREQc\nWmhoOOnpaQQFBdOyZUsAhg2L4de//iU7d+5gzJjb8ff3Z9asD+2c1Ll4enry1ltvXbT8H//4hx3S\niIjI9aisqmLTzuojccdOnMJoMDC4WxBjBoYS4OvR6HmaRZkDCA20MHVUFLN+SOO9eTuIn9ob13q4\ncFxEpKny9fUlIeH7OsuCgoL59NMva+6PGlV9Ufe500giIs4fPYyIaM8773yAiIiIs6usqmJjajbf\nrT9Idv5pTEYDQ3sEcdvNYfjb8SDRNZW5V155hZSUFAwGA/Hx8XTv3r3msZiYGAIDA2uukXj99dcJ\nCAjgL3/5C5s3b6aiooL/+Z//YdSoUfz6178mNTW15hveRx55hGHDhtX/q7qMIT2C2ZtZwJptWXyx\nZDcPju7caPsWERERERHnUlFZxYYdx/huw0FyT57BZDQwrGcwtw0IpbUDnOl31TK3adMmDh06xJw5\nc9i3bx/x8fHMmTOnzjoffvghnp7nzw3duHEje/bsYc6cOeTn5zNx4kRGjRoFwC9/+Uu7XuA9ZWQU\nh7KLWJ2SRWSwD0N6BNsti4iIiIiIOJ6KyirWbc/i+w2HyCs4g4vJwPBeIdw2IJRWPu72jlfjqmVu\nw4YN3HrrrQBERkZSUFBAcXExXl5el31Ov379ao7eeXt7c/r06YsuorcXs6uJ6RO78fKsJGYv2U27\nAAuhgY45T4WIiIiIiDSe8ooq1m7PYsGGgxwvLMXFZGRE7zaMHtAOP2/HKXHnXLXM5eXlER0dXXPf\nz8+P3NzcOmVu5syZZGZm0qdPH5599llMJhMeHtUXAM6dO5ehQ4fWnIY5e/ZsZs2aRatWrXjhhRfw\n8/O77L7ra0JUqDs/g9Vq4bn7+/C7jxN5/9tU/vqLW/ByoAnFHXUSxEtR1objTHmVtWE4U1Zwvrwi\nIiLnlFdUsjoliwUbD5FfVIqri5Fb+7Zh9E2h+Frc7B3vsq57AJQL5wx6+umnGTJkCD4+PkyfPp1F\nixYRFxcHwNKlS5k7dy7//Oc/ARg/fjwtW7akc+fOfPDBB7zzzju8+OKLl91XfUyICpeeWDDM6snY\ngWF8t/4gf/40iSfv7IbRYP8JxR1xEsTLUdaG40x5lbVhOFNWqJ+8KoMiItLYysorWZVylB82HuJk\ncRlmFyOj+rVl9E3t8PFy3BJ3zlXLnL+/P3l5eTX3c3JysFqtNfcnTJhQc3vo0KHs3r2buLg41qxZ\nwz/+8Q8++ugjLJbqP9A333xzzboxMTE1c+vYy4TB4ew/WsDWvXks2HCIsQPD7JpHREREREQaXml5\nJau2ZPJD4mEKSsowuxqJu6kdcf3b4e3pOGfsXY3xaisMGjSIRYsWAZCamoq/v3/NKZZFRUU88sgj\nlJWVAZCUlESHDh0oKiriL3/5C++//37NyJUATz31FEeOHAEgMTGRDh061PsLuh5Go4Gf3x6Nn7cb\n89bsJ/XgCbvmERFxRnfdNY6SkhJ7xxAREbmq0rJKFiYeZsbf1/Pl8r2cKa/ktgGh/OXxgdwzvL1T\nFTm4hiNzvXv3Jjo6msmTJ2MwGJg5cyYJCQlYLBZGjhzJ0KFDmTRpEm5ubnTp0oW4uDj+85//kJ+f\nzy9+8Yua7fz5z39mypQp/OIXv6BFixZ4eHjwpz/9qUFf3LXw9jDz+ISuvDo7mffnp/LSQ/0c8uJG\nERERERH5ac6UVbA8OZOFiYcpPl2Ou9nE2IGhjOrXDq8WrvaO95Nd0zVzzz33XJ37nTp1qrk9bdo0\npk2bVufxSZMmMWnSpIu2ExwczFdfffVTcjaoyGAf7r21A7MX7+bvX+9gxpTeuJiuetBSRKRJe/jh\nKbzyyhsEBgZy7FgWv/nNs1it/pw+fZozZ87wzDO/okuXrvaOKSIiclmnSytYtjmDxUlHKD5dTgs3\nF24fFMbIfm3xdHfeEnfOdQ+A0lQN7xXC3swCNqZmM2f5XqaMjLJ3JBERuxo6dDjr1q3mzjvvYc2a\nVQwdOpzIyA4MHTqMzZuT+OKLT/njH1+zd0wREZGLnDpTztLNGSxJOkLJmQo83V2YMCScW/u0waMJ\nlLhzVObOMhgMTIvtxJHsYpZtziAyxJsBXQLtHUtEBICEvd+xJWd7vW6zl3837mg/9rKPDx06nHfe\n+T/uvPMe1q5dxZNPPsOXX37Ov//9OeXl5bi765R0ERFxLCVnylmSdIQlP2ZwurS6xN0xNIIRfdrQ\nwq3pVZ+m94pugJvZxPQ7uvG7T5L45Ic02lq9CLFefnJ0EZGmLCIikuPHc8nOPkZRURFr1qykdWt/\nXnjh96Sl7eSdd/7P3hFFREQAKD5dzuKkIyzbfITTpZV4tXDlrmGRDO8V0iRL3DlN95X9RIF+Hjwy\npjPvztvBO/N28OK0vk36B0BEnMMd7cde8ShaQ7n55sF88MF7DBlyCydP5hMZWT0K8apVK6ioqGj0\nPCIiIrUVnSpjcdIRlm7OoLSsEm8PV8YND2dYr2DczU3/M3zTf4U/QZ+O/sT1b8fCTYeZtWAXj0/o\nisEBJhQXEWlst9wynMcee5hPPvk3Z86c5g9/mMmKFUu58857WLp0Md9//429I4qISDNUWFLGok2H\nWZ6cSWl5JT6eZiYODueWXiG4uZrsHa/RqMxdxp3DItifVciP6bksSTrCqP7t7B1JRKTRde4czapV\niTX3v/hibs3twYNvAWDMmNvx9PTk1KmiRs8nIiLNS0FxKd9sOMSC9QcoK6/Cx8vMHbdEcEuPYMzN\nqMSdozJ3GSajkcfGR/PyrCT+s2IfYUHeRLVtefUnioiIiIhIvTpZXMoPGw+zcmsm5RVV+FrcuHtY\nKEN7BOHq0vxK3Dkqc1fQ0suNxyd05S//2sLfv97BSw/1w8fLzd6xRERERESahfyiUhZsPMSqrUep\nqKzCz9uNSSM70jPcD1cXzQutMncVUW1bcvfwSOYs38vf56fyq3t7YjLqB0dEREREpKGcKDzD9xsP\nsSblKBWVNlp5uzNmYCiDuwURFOhDbq5O7QeVuWsyql9b9mYWsDk9l69W7eee4e3tHUlEREREpMnJ\nKzjNgg2HWLMti8oqG6193Bk7MIyBXQNxMemAyoVU5q6BwWDg4ds6k5FbwsLEw0QGe9Ono7+9Y4mI\niIiINAm5J0/z/YaDrNt+jMoqG/6+LRh7cxgDogNU4q5AZe4atXBzYfrErvzhsx/5+PtdhFi9CPTz\nsHcsERERERGnlZN/iu/WH2L9jmNU2WwE+HkwbmAoN3UJ0KVN10Bl7jq0sXrxYFwnPvh2J+/O287z\nU/viZm6+o+eIiIiIiPwUx06c4rv1B9mYmk2VzUZQKw/GDQyjf+cAjEbN73ytVOau04DoQPZlFrIs\nOYPPFqXx6NgumlBcREREROQaHM0r4bsNB0ncmY3NBiGtPRk3KIy+Hf1V4n4ClbmfYNKI9hw4VsiG\n1Gzah/gwvHcbe0cSEREREXFYmbnFfLv+IEm7crABbaye3D4onN4drRh1YOQnU5n7CVxMRp6Y0JWX\nZiXxr6V7CA30JiLY296xREREREQcSkZOMd+sP8jmtOoS187fi3GDwukV1Volrh6ozP1Eft7u/M/4\naN78civvfb2dmQ/2w+JhtncsERERERG7O5xdxLfrDrJ5dy4AoYEWbh8URs/2rXWJUj1SmbsB0WF+\nTBgawbzV+/ngm1SeuaenzvUVERERkWbr4LFCvl13kC178gAID/Jm/OAwukW0UolrACpzN2jMzaHs\nzywgZd9x5q89wMShEfaOJCIiIiLSqA5kFfLN2gOk7DsOQGSIN+MHhRMd7qcS14BU5m6Q0WDg0XFd\neHlWEt+uP0hEsDc92re2dywRERERkQa3L7OAb9YdZPv+6hLXoY0Ptw8Op0uor0pcI1CZqwee7q5M\nn9iNP36+mY++28mLD/bD2rKFvWOJiIiIiDSIPRkn+WbdQVIPnACgY9uW3D44nE7tWqrENSKVuXoS\nGmhh6qgoZv2QxnvzdhA/tTeuLppQXERE6iopKWHGjBkUFBRQXl7O9OnT6dmzJ8888wwFBQUEBATw\n5ptvYjZrUC0RcTzph/P5Zt1Bdh3KB6BzqC+3DwqjYztfOydrnlTm6tGQHsHszSxgzbYsvliymwdH\nd7Z3JBERcTDz5s0jPDycZ599luzsbKZNm0ZMTAyDBw/mwQcf5J133iEtLY3u3bvbO6qICAA2m420\nwyf5dt0B0g6fBCA6zJdxg8KJatvSzumaN5W5enb/qCgOZxezOiWLyGAfhvQItnckERFxIL6+vqSn\npwNQWFiIr68vK1asYPbs2QA8+eST9ownIlLDZrOx81A+3649wO6MAgC6Rvhx+6Bw2of42DmdgMpc\nvXN1MfHExK68PCuJ2Ut20y7AQmigxd6xRETEQYwZM4aEhARGjhxJYWEh77//Pj/72c/497//zfr1\n62nfvj3PP//8VU+z9PX1wKUeTue3Wp3rb5Qz5VXWhqGsDedcXpvNxpb0XL5cks6ug9XXxPXtHMDk\nkVF0DPWzZ8QazvTeNmRWlbkGYG3Zgp+N68Jbc7fx3tfbefHBfni6u9o7loiIOID58+cTHBzMxx9/\nTFpaGvHx8ZSWljJo0CCefPJJnn/+ef773/8yZcqUK24nP//UDWexWi3k5hbd8HYaizPlVdaGoawN\nx2q1kJNTyPb9x/lm3UH2Hy0EoGf71owbFEZ4kDeAQ7wmZ3pv6yPrlcrgNZW5V155hZSUFAwGA/Hx\n8XXO44+JiSEwMBCTqfrbwddff52AgIBLPicrK4v//d//pbKyEqvVymuvvdZkL/Du0b41YweG8d36\ng3z07U6euqs7Ro3sIyLS7CUnJzN48GAAOnXqRE5ODoGBgfTq1QuAQYMGkZiYaM+IItLM2Gw2NqUe\n4/MFOzl4rLp49I6yMm5gmM4wc3BXLXObNm3i0KFDzJkzh3379hEfH8+cOXPqrPPhhx/i6el51ee8\n/fbb3HfffYwePZo333yTuXPnct9999X/q3IQEwaHc+Bo9YTiCzYcYuzAMHtHEhEROwsNDSUlJYXY\n2FgyMzPx9PTkpptuYuPGjQwYMIDU1FTCw8PtHVNEmoHyiko27cphyY9HOJxdDEDfjlbGDgyjXYBK\nnDO4apnbsGEDt956KwCRkZEUFBRQXFyMl5fXdT8nMTGRl19+GYDhw4fzz3/+s0mXOaPRwM9vj+bl\nT5KYt2Y/4cHeRIc5xnnGIiJiH5MmTSI+Pp7777+fiooKXnrpJTp27Mhzzz3H22+/TevWrXniiSfs\nHVNEmrC8gtOs3HKU1SlHKT5djsEAg3sEM6pvG9pYL/8ZXxzPVctcXl4e0dHRNff9/PzIzc2tU+Zm\nzpxJZmYmffr04dlnn73sc06fPl1zWmWrVq3Izc2tz9fikCweZh6f0JVXZyfz/vxUXnqoH37e7vaO\nJSIiduLp6clbb7110fJ//vOfdkgjIs3FuZEpl2/OYOvePGw28GrhyugB7RjeM4TOHfyd5jo0Oe+6\nB0Cx2Wx17j/99NMMGTIEHx8fpk+fzqJFi676nMstu1B9jdQF9h3xxmq18LPicv6RsI0Pv9/Fn54Y\njKuL8YrrOwtlbTjOlFdZG4YzZQXnyysi0hycLq1g/Y5jLE/OIOt49cBJoYEWRvRuQ//O/phd6+ez\nttjHVcucv78/eXl5NfdzcnKwWq019ydMmFBze+jQoezevfuyz/Hw8ODMmTO4u7uTnZ2Nv7//Ffdd\nHyN1gWOMeNOvQyu2RgewMTWbd+dsYcqoqEuu5whZr5WyNhxnyqusDcOZskLDj9YlIiLXJzOvhOXJ\nGazfcYzSskpcTAZujg4gpk8bIoK8MWhgvibh8oeHzho0aFDN0bbU1FT8/f1rTrEsKirikUceoays\nDICkpCQ6dOhw2ecMHDiwZvnixYsZMmRIg7woR2QwGJgW24kQqyfLkjPYmHrM3pFEREREpAmprKpi\nc3our/17Cy98lMiK5Ew83FyYODSC154YxM/GRRMZ7KMi14Rc9chc7969iY6OZvLkyRgMBmbOnElC\nQgIWi4WRI0cydOhQJk2ahJubG126dCEuLg6DwXDRcwCeeuopZsyYwZw5cwgODq5zVK85cDObmD6x\nG7/7JIlPFqbR1t+LEF1kKiIiIiI3oPBUGWtSjrJiSyYnCksB6NSuJSP6tKFnh9aYjFc9fiNO6pqu\nmXvuuefq3O/UqVPN7WnTpjFt2rSrPgeqT9mcNWvW9WZsUgL9PHhkTGfenbeDd+bt4MVpfWnhprnb\nRUREROT67D9ayLLNGSSlZVNRacPN1cTwXiHE9A7RAYNmQi3CDvp09CeufzsWbjrMrAW7eHxCVx3u\nFhEREZGrOjc33PLkDA5kVV+rHODnQUzvEAZ1DcLDXR/vmxP927aTO4dFsD+rkB/Tc1mcdITY/u3s\nHUlEREREHNRFc8MBPdu3ZkSfNnQO88WoAwPNksqcnZiMRh4fH81Ls5L474p9hAd5E9W2pb1jiYiI\niIiDsNls7DqUz7Jac8N5urvUzA3XumULe0cUO1OZsyMfLzcen9CVv/xrC3//egcvPdRPQ3OLiIiI\nNHOXnBsuwMKIPpobTupSmbOzqLYtuXt4JHOW7+Xv81P5y1PNZ7oGERERETnv6Nm54dadnRvOZDw7\nN1zvNkQEa244uZjKnAMY1a8tezML2Jyey2cLdjF2gK6fExEREWkOKquq2LrnOMuTM9h1KB8AX4sb\ntw0IZWiPYHw8zXZOKI5MZc4BGAwGHr6tM5m5JSSs3EuQrzt9OvrbO5aIiIiINJDLzQ0X07sNvaI0\nN5xcG5U5B9HCzYXpE7vyh8838/H3uwixehHo52HvWCIiIiJSj/YfLWR5cgabdmluOLlxKnMOJMTq\nxZN39+SNLzbz7rztPD+1L25mXeAqIiIi4sw0N5w0FP3kOJhhvduwdVc2y5Iz+GxRGo+O7aKLXUVE\nRESc0PGCM6zcmsmqrXXnhovpE0KXMD/NDSc3TGXOAU0a0Z4DxwrZkJpN+xAfhvduY+9IIiIiInIN\nzs0Nt+77XWzckXV+brib2jGsVwhWzQ0n9UhlzgG5mIw8MaErL81K4l9L99Au0EJksI+9Y4mIiIjI\nZWhuOLEHlTkH5eftzmPjo3ljzlbem7eDmQ/1w9tDQ9OKiIiIOJJzc8Ot33GMM2fnhhsQHcCdI6Lw\na+Giy2WkQanMObAuYX5MHBJBwur9fPhNKs/c0xOjUf9BEBEREbGnyqoqUvYeZ9nmunPDja41N5zV\naiE3t8jOSaWpU5lzcLfdHMq+zAJS9h1n/toDTBwaYe9IIiIiIs3SubnhVm7J5LjmhhMHoDLn4IwG\nA4+O68LLs5L4dv1BIoK96dGnaxXVAAAgAElEQVS+tb1jiYiIiDQbB7IKWbY5g027cqiorKqZG254\n7xDaaG44sSOVOSfg6e7K9IndeGX2Zj76bicvPthPIyGJiIiINKDyikqS0nJYtjmTA1mFAAT4tiCm\nTxvNDScOQz+FTiI00ML9o6KYtSCNd+dt57dT++DqolGRREREROrTubnhVqccpeiU5oYTx6Yy50SG\ndA9mX2YBq1Oy+GLJbh4c3dnekUREREScns1mI+1QPsuSM9myJ1dzw4nTUJlzMlNGRnHoWDGrU7KI\nDPZhSI9ge0cSERERcUqXmxsupk8IN3UO0Nxw4vCafJmrslVhs9nsHaPeuLqYeGJiV16elcTsJbtp\nF2AhNNBi71giIiIiTiPreAnLN2eybkdWnbnhRvRuQ0Swt+aGE6fRpMtcWWUZz697BQ+zO91bdaWX\nf3fCvNs6/S+otWULfjauC2/N3ca787Yz86F+eLq72juWiIiIiMOqqrKRsjePZckZ7DxYa264m9ox\ntGcIPp5mOycUuX5Nusy5GF3oE9CTpJxklh1ZzbIjq/F1a0kv/2709u/h1MWuR/vWjBsYxrfrD/LR\ntzt56q7uuiBXRERE5AJFp8pYfZm54Xp2aI2LSXPDifNq0mXOaDAyqeME/ufmyazZnUxyzja25e5k\n+ZE1LD+yplax606YdzunK3bjB4ez/2j1hOILNhxi7MAwe0cSERERcQgHsgpZvjmDxFpzww3rFUKM\n5oaTJqRJl7lzXE2udGvdhW6tu1BeVUHaid1sydnOtrzUi4rduVMxjQbH/5bGaDTw89ujefmTJOat\n2U94sDfRYX72jiUiIiJiF+UVVSSlZWtuOGk2mt1PtKvRpU6xSz+xp/qIXa1i19LNp86pmI5c7Cwe\nZh6f0JVXZyfz/vxUXnqoH37e7vaOJSIil1FSUsKMGTMoKCigvLyc6dOn88EHH3Dq1Ck8PDwAmDFj\nBl27drVzUhHncaLwDCu2aG44aX6aXZmrzdXoQtfWnenauvNFxW7FkbWsOLK2VrGrPhXTEYtdZLAP\n997agdmLd/P3r3cwY0pvnf8tIuKg5s2bR3h4OM8++yzZ2dlMmzYNq9XKn/70J6KiouwdT8Rp2Gw2\n0g6fZPnmDJJrzQ0Xd1M7hmtuOGkmrqnMvfLKK6SkpGAwGIiPj6d79+4XrfPGG2+wdetWPv/8c/77\n3//yzTff1Dy2Y8cOtmzZwtSpUx32m8faxa6iqoK0E3vYkrOdlAuLnbX6VMxwH8cqdsN7hbAvs4AN\nqdnMWbaXKaP0gUBExBH5+vqSnp4OQGFhIb6+vnZOJOJcTpdWsCH1GMuTMzmaVwJAuwAvRvRpo7nh\npNm5apnbtGkThw4dYs6cOezbt4/4+HjmzJlTZ529e/eSlJSEq2v18Ph33303d999d83zf/jhh5p1\nneGbR5daxe7eqgrS8/eSnL2tuthlrGVFRnWx62mtnu4gwifU7sXOYDDwQGwnDucUsyw5g8gQbwZE\nB9o1k4iIXGzMmDEkJCQwcuRICgsLef/993njjTd4++23yc/PJzIykvj4eNzddcq8SG15BadJWHuA\npZsOn58brksAMX3aEKm54aSZumqZ27BhA7feeisAkZGRFBQUUFxcjJfX+VGAXn31VZ555hneeeed\ni57/7rvv8vrrr9dj5MblYnQhulUnolt1Ol/scraxLTeVlRnrWJmxDh+zNz3Pnoppz2LnZjYxfWI3\nfvdJEp8sTKOtvxchGq1JRMShzJ8/n+DgYD7++GPS0tKIj4/n8ccfp2PHjrRr146ZM2fyxRdf8Mgj\nj1xxO76+Hri43PgRCKvVcsPbaEzOlFdZ60dFZRXzV+3jX4vTKSuvpJWPO3fFdGDUgFB8LY79pYcj\nv6+X4kx5lbXaVctcXl4e0dHRNff9/PzIzc2tKXMJCQn079+fkJCQi567bds2goKCsFqtNcuu55vH\n+vpDBfX3JgYF9GNYp35UVFawIyedDUeS2ZS5lVUZ61iVsQ7fFj7c1KYXN7ftTcfWkT+p2N1IVqvV\nwjP39uZPnybxj29SefMXt+DRgBOK6xep4ThTXmVtGM6UFZwvr70kJyczePBgADp16kROTg4xMTGY\nTNV/72JiYliwYMFVt5Off+qGs1itFnJzi254O43FmfIqa/3Ym1nAZwvTyMgtwdvDlSfu7E6Xtj64\nmIxUnCkn90y5vSNeliO/r5fiTHmbW9Yr/X297gFQbDZbze2TJ0+SkJDArFmzyM7OvmjduXPnMnHi\nxJr7DzzwwHV981gff6ig4f6Fh7i0467wdkwMHVfniN3CPStZuGclPmYLPf270cvanciWYddU7Ooj\na4cgC3H927Fw02H+8lkST0zo2iCnHjS3X6TG5Ex5lbVhOFNWaPg/Vk1JaGgoKSkpxMbGkpmZiYeH\nB4888ghvv/023t7eJCYm0qFDB3vHFLGrU2fKmbtqP6u2ZGIDhvYI5q5hkYS383Oq/zaKNLSrljl/\nf3/y8vJq7ufk5NQcadu4cSMnTpxgypQplJWVcfjwYV555RXi4+MBSExM5Pnnn6957siRI2tuX+s3\nj87AZDTRpVVHurTqSGXHO0jP38uWnG2k5KayKmM9qzLW42220NNafSrmtRa7G3HnsAj2ZxWyOT2X\nxUlHiO3frkH3JyIi12bSpEnEx8dz//33U1FRwcsvv0x+fj4PPvggLVq0ICAggKeeesreMUXswmaz\nkZSWw7+W7qGwpIyQ1p5Mje1IVNuW9o4m4pCuWuYGDRrE3/72NyZPnkxqair+/v41p1jGxcURFxcH\nQEZGBr/5zW9qilx2djaenp6YzWag+pfzoYceavLfPNYudpM73sHu/H0k52wjJW8HqzPXszqzdrHr\nRmTL8AYpdiajkcfHR/PSrCT+u2If4UHe+g+hiIgD8PT05K233rpo+W233WaHNCKOI+fkaWYvSmfH\ngRO4uhi585YIYvu303RLIldw1TLXu3dvoqOjmTx5MgaDgZkzZ5KQkIDFYqlzpO1Cubm5+Pn51dw3\nGAzcc889zeqbR5PRROdWUXRuFcXkqonsPrmPLTnb2Jp7YbGrHhWzfcvwet2/j5cbj0/oyl/+tYW/\nf72Dlx7qh4+XW73uQ0RERORGVFRWsWjTYb5Zd5Dyiiqiw/2YOioKf18Pe0cTcXgGW+2L4BxMfZ0T\n7WjXnlRWVbLn5H6Sc1JIyU2luLx6jhSL2Yub2/ams3dn2tfjEbtFmw4zZ/leotq25Ff39sRkrJ/t\nOtr7eiXOlBWcK6+yNgxnygq6Zs4e6uPnozn+nDUWZb02ezJO8tnCdDLzSvD2NHPviA707+x/2Wv9\n9b42HGfK29yy1usAKHLjTEYTnfw60MmvA5OiJp4tdttIyd3B4n2rWcxqLGavmlMx27eMuKFiN6pf\nW/ZlFvBjei5frdzPPTHt6/HViIiIiFyfkjPlzF25j1VbjwIwrFcId90S0aAjcIs0RSpzdla32E0g\nl2Os2L2Rrbk7WJO5gTWZG7C4etHDvyu9rdWnYpqM1zddg8Fg4KHbOpORW8LCTYeJDPGmT0f/BnpF\nIiIiIpdms9lI3JnNl8v2UHiqnBCrJ9NiO9G+jY+9o4k4JZU5B2Iymuhm7USgMYR7oiaw9+QBknNS\n2Jq7g7WZG1mbuREvV8+aa+w6tIy45mLXws2F6RO78vvPfuTj73cRYvUi0E/noouIiEjjyMk/xeeL\n0kk9mI/ZxchdwyIZ1a+tBjgRuQEqcw7KZDTR0a89Hf3any92udtIydnB2qOJrD2aiJerJz2sXel9\njcUuxOrFg3Gd+ODbnbw7bzvPT+2Lm7l+JmUXERERuZSKyip+SDzMd+urBzjpGuHH1FEdsbZsYe9o\nIk5PZc4J1C52k6ImsPfkfpJztrM1Zzvrjiay7jqK3YDoQPZlFrIsOYPPFqXx6NguDTKhuIiIiMju\nIyf5dGEaWcdP4eNp5t4xHejX6fIDnIjI9VGZczJGg5Eo3/ZE+bbnnqjx7D15gC0529iSe2Gxi6aX\nf3eiWkZeVOwmjWjPwWOFbEjNJjLEh5jebez0akRERKQpKj5dztyVe1mdkoUBGN47hDuHRuLhro+e\nIvVJv1FOrLrYRRLlG8ndUePZd/IAyTXFbhPrjm7C09WDHq270jvgfLFzMRl5fEJXXpqVxL+X7iE0\n0EJksC48FhERkRtjs9nYmJrNl8v3UHSqnDZWL6aN7qjPGSINRGWuiTAajHTwjaRDnWK3na2521mf\ntYn1WbWKnX93onwjeWx8NG/M2cp783Yw86F+eHuY7f0yRERExEllnzjFZ4vS2XUoH7OrkXuGt+fW\nvm00wIlIA1KZa4LqFrvb2XfyIFtyt7E1p1axc/GghzWawQMDWLP+NB9+k8oz9/TEaNQ57CIiInLt\nyiuq+CHxEN+tP0RFZRXdI1tx/6goWvtogBORhqYy18RVF7sIOvhGcFeH29lfcKh6uoOc7azPSgLA\ns4+Z3cetfLy6gIeHDrnueexERESkeUo/nM+nC9M5duIUPl5mptwaRZ+OVg1wItJIVOaaEaPBSPuW\n4bRvGV6r2G0jOXsbVdZMtlZl8qvVy+kV0JXe/j3o5NtexU5EREQuUny6nP8s38va7dUDnIzo3YaJ\nQyM0wIlII9NvXDNVt9iNY+OBXXy+aRWlvllszPqRjVk/4uHSgu7WaHr7d6ejb3tcjPpxERERac5s\nNhvrdxxjzvK9FJ8up52/Fw/EdSIi2Nve0USaJX06F4wGIwMjoqks9mXWgl0EtSule+8yUvJ21BS7\nFi4t6NE6ml7+3ejk10HFTkREpJnJOl7C54vSSTt8ErOrkUkx1QOcmIwa4ETEXvSJXGoM6R7MvswC\nVqdk0cE3nD/E3c6BgsM189htPPYjG49VF7vurbtwc3gvTGVmLK4WvN0suJk0GqaIiEhTU15RxYKN\nh/h+w0EqKm30bN+aKSOjaOXjbu9oIs2eypzUMWVkFIeOFbM6JYvIYB+G9AgjsmUYd3QYy8HCw9Xz\n2OVsJ/HYZhKPba7zXLPJjLerF95uFixmCxazF95mC95mLyxmS53bKn4iIiKOb9ehfD5blE72iVP4\nWty479Yoeke11gAnIg5CZU7qcHUx8cTErvzukyRmL9lNuwALoYEWjAYjET5hRPiEcUf7sRwsPEJu\n5TGyThynsKyIorLis/8s4mDhEapsVVfcj5vJfLbgVRe+K5U/s4qfiIhIoyo6VcZ/lu9l3Y5jGAxw\na982TBwSQQs3fXQUcST6jZSLWFu24GfjuvB//93Gu/O2M/Ohfni6u9Y8Xl3sQrnJ2pXc3KKLnl9l\nq6Kk/FRNwTtX9mrfP7fsQMFhbNiumMfNZK4pfOfL38VH+1T8REREbozNZmPt9iz+s3wvJWcqCA2w\n8EBcR8KDNMCJiCNSmZNL6h7ZmnEDw/h2/UE++nYnT93VHeM1nlJhNBjPli0vggm84rrnil/to3t1\nj/SdX5ZXcOKqxc/d5FZzhK92+QsubA2lrnXKn9nkesVtiYiINCdZx0v4bGE66UdO4mY2ce+IDsT0\nCdEAJyIOTGVOLmv84HD2Hy0gZd9xFmw4xNiBYfW+j9rF72pqF79Llr/SIorKiy9d/A5cvD13k/vZ\nYndx+bvwyJ+rip+IiDRRZeWVzFu9nwUbD1FZZaNXh+oBTvy8NcCJiKNTmZPLMhoN/Pz2aF7+JIl5\na/YTHuxNdJif/fLUKn4hBF1x3SpbFcXlJTUlD/cKMo/nUlhaRGFZMUW1SmDu6ePXcMTPHW83r+qR\nO81nB3mpfdt8/jEVPxERcRY7D57gX0sTOZpXgq/FjftHRtErymrvWCJyjVTm5IosHmYen9CVV2cn\n8/78VF56qJ9TfFNnNBjPHlWzEOIVhNVqIdfz4uv7ACqrKikuP0XRNZzqmXvq6sWvhYv7RUf7qqdv\nOH/U79xjrpqvT0RE7KCwpIw5y/ewITUbowFG9WvL+MHhGuBExMnoN1auKjLYh/tu7cDni3fz3tc7\n+PWU3riYms758yajCR83Cz5ulquue674nRu580rl71qL34WndVpcvWh53IPi4tKa9a5lCGgDhgvu\nX7jChY9feZsXPX7R3eoFXifdKC4uvczWLtjnxaGutItrznD5LdRdYilyp7DoDGCr/ndjq7lVvcx2\n9p/n/v/Zf3211gBbrcepHiyAWutfvD1q9le9+Tpbu+z2Whx1peRU6bmnX5DharnPP3Z2Fxe9rtrb\nO/943Qx1ll4ht6vRhQf63oErHhe9/yLiWKpsNtZuy+K/K84OcBJo4Rf39sbHzWTvaCLyE6jMyTUZ\n1iuEvZkFbEjNZs6yvUwZFWXvSHZx/cWv5GzZq3tqZ+1lRWXFZJ/KbYT0Ig3DgIGYgpsJd4u0dxQR\nuYLMvBI+X5jG7owC3M0m7ru1AzG92xAQ4H3J0alFxPGpzMk1MRgMPBDbicM5xSxLziAyxJtxw65e\naJqz6uLnjY/b1YdzrqyqpKi8uGYKB4u3GwUFp4HzR1CupPaRlOrnXLTGlR+/yvOvlMHb270ma2Pt\ns/rp1/C+XLANi8WdoqIzNUf0zh3xNJxbYqh5pM5j1f+84PELnn9+e2cfw0D1/50/NmjAcPbo5KX3\nU/sxX19PTp48ddHzz2/z8hnqLDHUyn/2lV6c4cLXcH4f5zZRZ6uGWu8ZBkxGE20CW+nDoIiDKiuv\n5LsNB/lh42Eqq2z0ibJy760dnOKyCRG5MpU5uWZuZhPTJ3bjd58k8cnCNLp3CsDDdG3TFciVmYwm\nWrr50NLNB6D6Gj9X5/hgbLVayHV3oqxOUjisrSzkVjlHVhFxXKkHTvD5onRyTp6mlbcbU0Z2pGeH\n1vaOJSL1pOlc+CSNItDPg0fGdKasvIo//nMTB7IK7R1JRERELlBQUsYH36Tyxpyt5BWcIbZ/W37/\n6E0qciJNzDUdmXvllVdISUnBYDAQHx9P9+7dL1rnjTfeYOvWrXz++eckJiby//7f/6NDhw4AREVF\n8cILL5CVlcX//u//UllZidVq5bXXXsNsNtfvK5IG16ejP7cPCuObdQf5w6c/Mrx3CHcMjcTDXQd6\nRURE7KnKZmN1ylHmrtjHqdIKwoO8mRbXkXYBujRCpCm66qfvTZs2cejQIebMmcO+ffuIj49nzpw5\nddbZu3cvSUlJuLqen1+rf//+vP3223XWe/vtt7nvvvsYPXo0b775JnPnzuW+++6rp5cijWnCkAhu\n6h7M3+ZsZXlyJpvTc5k8ogP9O/tf08iLIiIiUr8ycov5bGE6ezOrBziZMjKK4b1CMBr1d1mkqbrq\naZYbNmzg1ltvBSAyMpKCggKKi4vrrPPqq6/yzDPPXHVniYmJjBgxAoDhw4ezYcOGn5JZHET39lZe\nfrg/E4dGcKq0gvfPns6RfeKUvaOJiIg0G6XllcxduY+XZyWxN7OAvp38+ePPBjCiTxsVOZEm7qpl\nLi8vD19f35r7fn5+5OaeH0Y9ISGB/v37ExISUud5e/fu5bHHHuPee+9l3bp1AJw+fbrmtMpWrVrV\n2Y44J1cXI+MGhvH7R2+ia4QfOw/m88LHm5i/9gDlFZX2jici4nBKSkp48sknmTp1KpMnT2bNmjU1\nj3355ZfExMTYMZ04m+37j/PCR4ks2HiIll5u/L+7uvPEhK74WtzsHU1EGsF1X+RUezjwkydPkpCQ\nwKxZs8jOzq5ZHhYWxpNPPsno0aM5cuQIDzzwAIsXL77sdi7H19cDF5f6mcTSanWec8WdMavVauGV\n9lbWb8vig6+3M3/tAZLScnj8zu70jPK3c8pqzvS+gnPlVdaG4UxZwfny2su8efMIDw/n2WefJTs7\nm2nTprFw4UKOHz/OkiVL7B1PnERBcSn/XraHTbtyMBoMjL6pHbcPCsfNrMm/RZqTq5Y5f39/8vLy\nau7n5ORgtVoB2LhxIydOnGDKlCmUlZVx+PBhXnnlFeLj47ntttsAaNeuHa1btyY7OxsPDw/OnDmD\nu7s72dnZ+Ptf+UN+fn79nK7nVMORO3nWqGALv3+kP/PW7GfZ5gxeeH8DN3UJYHJMe3y87PctoTO9\nr+BceZW1YThTVqifvM2lDPr6+pKeng5AYWFhzdkvr732Gk8//fQ1XbYgzVeVzcaqrUeZu3Ifp0sr\niAj2ZlpcJ9r6e9k7mojYwVXL3KBBg/jb3/7G5MmTSU1Nxd/fHy+v6v9gxMXFERcXB0BGRga/+c1v\niI+P55tvviE3N5dHHnmE3Nxcjh8/TkBAAAMHDmTRokWMHz+exYsXM2TIkIZ9dWIXLdxcuO/WKAZ1\nDeKzRWkk7sxm277j3HlLBMN66kJsEWnexowZQ0JCAiNHjqSwsJD333+fxMRE3Nzc6NGjh73jiQPL\nyCnm04Vp7DtaSAs3F6bGduSWnsEYNfCYSLN11TLXu3dvoqOjmTx5MgaDgZkzZ5KQkIDFYmHkyJGX\nfE5MTAzPPfccy5Yto7y8nJdeegmz2cxTTz3FjBkzmDNnDsHBwUyYMKHeX5A4jtBAC7+d2pdVWzOZ\nu2o/sxfvZt32LB6I7URoYPP4Bl5E5ELz588nODiYjz/+mLS0NH7zm9/g4eHBe++9d13bqa9LEZzt\niKgz5a2vrGdKK/hySTrzVu2jqsrGkJ4hPDq+K37e7vWyfWie72tjcKas4Fx5lbWawXYtF6/ZSX2d\nYuRMpys11awFxaXMWbGXjanZGAwwoncbJg6NoIVb48xN50zvKzhXXmVtGM6UFXSa5fWYOXMmAwcO\nJDY2FoDu3bsTGBhIy5YtAdi5cycjR47kr3/96xW3Ux8/H83x56yx1FfWbfvymL14N3kFZ2jt487U\n2I50i2hVDwnPa47va2NwpqzgXHmbW9Yr/X3ULM/SKHy83Pj5uGgGdwvi80XpLN2cQVJ6DvfdGkXf\njlbNTScizUZoaCgpKSnExsaSmZlJUFAQixYtqnk8JibmqkVOmr78ouoBTn5My8FkNHDbgFDGDQrD\nzVUDnIjIeSpz0qi6hPnxu0f688PGw3y34RB//3oHXSP8uH9kFP6+HvaOJyLS4CZNmkR8fDz3338/\nFRUVvPTSS/aOJA6kqsrGyq2ZfLVqH6dLK2kf4sMDsR1powFOROQSVOak0bm6mLh9cDg3RQcwe1E6\nO/af4IWPNzH25lDibgrF1eWq0x+KiDgtT09P3nrrrcs+vnz58kZMI47kcHYRny5M50BWIR5uLjwQ\n15GhPTTACUBZZTlZJcc4XF6J7bQJb7MFi9kLF6M+ykrzpt8AsZsAXw9+OaknSWk5/HvpHuatOcCG\n1Gymxnakc6jv1TcgIiLSBJSWVTJ/7QEWJx2hymZziCl97MVms1FYVkRGcRaZRUfJKD5KZnEW2ady\nsXHxMA9erp54my14my34uHlX33az4HN2mbebNz5mC+4u9TdYjIgjUZkTuzIYDPTvHEDX8FbMW72f\n5ckZvPbvLdwcHcA9MR3w8TTbO6KIiEiD2bo3jy8Wp3O8sBRry+oBTrqG1+8AJ46qsqqS7FO5ZBSf\nLW1FWWQWZ1FUXlxnPXeTOxE+YbSxBBHi58+x/OMUlhVRUFpIYVkx+aUnOVpy7Ir7MpvM1YXvbMG7\n1G0fN288XT0wGnSGkDgPlTlxCB7uLkwZFcXAboF8tiidDanZpOw9zl3DIhmqOXRERKSJyS8q5V9L\nd7M5PReT0cCYm0MZNzAMcxMd4ORU+Wkyi4+SUZxVc7QtqySbiqqKOuu1cvelh080IV5BhFiCaeMV\nTCt335qB0i43MmBZZRmFZcUUlhVSUFpEQVkhRaVFFJTVvb2/4NAlj/CdYzQYsbh64eN29sie2fv8\n7bNH+c4dCXQ1udbvmyTyE6jMiUMJD/LmhQf6smJLJgmr9/HZonTWbc9iamxH2gU0j2HLRUSk6aqq\nsrE8OYOE1fs5U1ZJ+zY+TIvtSIi1aQxwUmWr4sSZfDKKqotb5tnyduJMfp31XIwuBHsG0MYrmBCv\nYNpYggnxCqSFS4uftF+zyUzrFn60buF31XxFZSVnS19hnQJYWFZUczurJJvDRZlX3JaHS4u6Je9s\n6fM5WwBL3QKpLDfRwsVdo3ZLg1GZE4djNBoY0acNfTpa+XLZHjbtyuHlT5IY2bct4weHN9rcdCIi\nIvXp0LEiPl2YxsFjRXi6u/Dg6E4M7h7ktGefnBuU5Hxxqz7idqaytM56FlcvOvtFnS1uQYR4BRHg\nYcVkbPyjkEaDER83Cz5uFtpaQi67ns1m40zlmbMlr5DCWkf5CkvPFsCyIopKizh2KueK+3Q1ulx8\nlK/OEb/qAujl6mmX90Scmz4Vi8Nq6eXGY+O7Mrj7cWYv2s3ipCMkpeVw74gO9NHcdCIi4iTOlFXw\n9ZoDLPnxCDYb3BwdwKSYDng7yXXh1zooiQEDAZ7+tPEKqlXcgvFxc74zawwGAy1cWtDCpQWBnv5X\nXLe8qoLCC47sFZYVUWY8Q3bB8bPLizhUdISqwqrL7xMDXmZPfMzedUrehUf8vM0WzCbn+NmRhqcy\nJw6va3grfvdIfxZsPMSCjYd47+sddI9sxZSRUVhb/rTTMURERBrDlj25fLFkNycKS/H3bcHU2I5E\nh135VEB7+imDkpwrbkGegZib4XVkrkYXWrXwpVWLuiNxX3h9X5WtipLyU7UGbyk6e8Sv+ihf4dmj\ngDmn88goPnrFfbqb3Gsd5as1kucFo3p6unjoy+/rVGWrqvlfZa3bdZdVXvT4heueWy/KrR1uNNxp\n1Cpz4hTMriYmDIlgQHQgny9KZ9u+46QdSmTcoDBi+7fDxaSRp0RExHHk5p/mb19tY8uePExGA2MH\nhjH25lCHGuDk3KAkSfknSDt24CcPSiLXxmgwYjF7YTF7EeIVdMV1z1SUXnAtX60CWOt29qncK27H\nZDBd5ihf3QLobbZgs9morLpcSbl4+cUlpvKi59l+8raqqDy73qXWdTUbOXWmrEH2e6UBcn4KrzRP\nXh30YoP9vqjMiVMJ9LaGRMgAACAASURBVPPguck9SdyZzZfL9vDVqv2s33GMB2I70rGd5qYTERH7\nstlsLE+uHsTrdGklUW18eCCuE8GtPe2W6cJBSc6dJnm1QUnOXd/m4aqzYBqbu4sb7i5W/D2sV1yv\nsqqypuDVLnnnj/RVL8ssOsoh25FGSm9fBgyYDEaMBiNGg6nW7er/uRhMGE2uGKm+bzq73rnHa9Y3\nGjFivOD5pjqP19zmgueeW89opHNwRIN+8aEyJ07HYDAwIDqQ7pGt+Gr1flYmZ/Lnf21hUNdA7o5p\nj7eHziMXERH7WLMtiy+W7Mbi4cpDozswqJEHOPkpg5J0Dg7HUuVrt0FJ5KczGU34urfE173lFdez\n2WyUVJyqU/BqF8CismJczSYqyqsuKji1y8yFxah62SWK0NnbhksVnCtu6/L7rb0t/9be5J84fclt\nOdoR48tNp1FfVObEaXm4uzJ1VEcGdQ3is4VprNtxjK1787h7eHunHh1MRESc0+HsImYv3o2nuwt/\nfWYYxsrKBtvXTxmUJKRmYJK6g5I09IdNsT+DwYCXqyderp4EE3jJdZzp56BlC8v/b+/eA6Kq8/+P\nP4f7XUFmuN9ERQVRMfGC4hUvaWa2lZVa39wuau3l22627Pa173c3266/zdoutuW2rbtZRmqZaZqa\nFwQVBUFFMQVE5KIooqCA8/vDlc3VRJNxZuD1+Is5M3PmdT7WfOZ9zud8PtS76BYbUDEnrUDHYB+e\nefAWvtleQtqG7/jrir1szCll2ugYQk2tY90eERGxbbVnG3hzSS4NjeeZeUccAX4eLfbD+MdMSnKx\ncGurk5KItBUq5qRVcHRwIKVvGLd0NfHPNfvZtrecZxdsZVTfMCYMirR2PBERacXMZjMLvtxDeVUt\nY/uH06uT/4/e18VJSb5/b5smJRGRH6JiTloVX29XZk6MI+fAMf6+Kp+vMovI3FvGjDt7Eh2gq3Qi\nItLy1mw/zLb8CrqEtmNScsdres/1TkoS4hV8yaLbmpREREDFnLRS8dEd+MNP+/FF+iFWbCniuQWZ\n9Orkz30pnfFvpw5QRERaxndHqln0TQHeHs48enscjg6X38dzvZOShHxv0W1NSiIiV6NiTlotF2dH\nJiVH0797IIvWHmBnQSW7C49ze1IUKX3DtDadiIjckJraet5aksv582YemRCLr7crjecb2X/iOzZV\nVpD/r7XbfsykJCIi10LFnLR6wf6ePDdjIMvW7WfRNwV8su4Am/OOMnVUDF3Crj6Vr4iIyJWcN5t5\n74vdHKuu47akcAzeFfx9zxpyKvI43XCm6XWalERELEnFnLQJBoOBgXFBxEf78+n6A6zfeYQ/Lsxi\nUHwQdw2Nxltr04mIyHVYkXGQXZV7MfaoYrP5W1ZnXyjg2rl4MyQ0ib4RcXif99WkJCJiUSrmpE3x\ncnfmgTFdSeoRxN++ymdjTik791dy17BoknpobToREflhDecbyK8qYN3BbeRV78E1pp4aoJ2DD0ND\nk+htiqdjuwgcDA52tWaXiNgvFXPSJnUKacec/7qF1dsOs2TDQRZ8uZdNOaVMHR1DiFGzXoqIyAUN\n5xvYe3w/O8p3kV2ZR21D7YUnzrvSq90tDI9OJKpdOA4G3YctIjefijlpsxwdHBidGE7frib+uXo/\n2/dVXFibLjGMCQOjcHXR7GEiIm3RxQIuqzyHnMrdTQVcexcfnM+EUXHIlzv69GFcn0jrBhWRNk/F\nnLR5fj5uzJrUg50FlSxctY8VW4rI3F3O/aO63NDCryIiYj8uLeDyqG2oA6C9azv6B/UhwRTPrpzz\nLM09RM/oDoztH2HlxCIiKuZEmvTq5E+3CF8+33SIlZlFzFucQ0IXI/eN7Iyfj5u144mISAurP9/A\n3uP72FG+65ICzte1PQOC+tLbFE+kTxgOBgd2HzrOsk076eDjyvTx3XWPtYjYhGsq5ubOnUt2djYG\ng4HU1FTi4+Mve80rr7zCzp07+fDDDwF48cUX2b59Ow0NDTz66KOMGjWKp59+mry8PNq3vzAd/PTp\n0xk6dGjLHY3IDXJ1duQnQ6MZEBvAhyvzydpXQd7B49w+KIqRt4RqbToRETt3sYDLKs8hp2I3dY2X\nFnAJpngi/lXAXVR16izzl+Xh4GDgsYlxeLlrWQERsQ3NFnOZmZkUFhayaNEiDhw4QGpqKosWLbrk\nNQUFBWzduhVn5wtfblu2bGH//v0sWrSIqqoq7rjjDkaNGgXAf//3fzNs2DALHIpIywkxejH7/gQ2\n7TrKx2sL+HhtAZtzjzJtTAydQtpZO56IiFyH+sZ69hzfR1b5LnZVXlrADQzuS4KpJ5E+YVdcQqDx\n/HneWZZH9Zl67h3Zmehg9QEiYjuaLebS09MZOXIkANHR0Zw8eZKamhq8vP49498f//hHfvnLX/LG\nG28A0Ldv36ardz4+PtTW1tLY2GiJ/CIWYzAYGBQfRK/O/nyytoANOaXM/XA7yT2D+cnQaJ2ZFRGx\nYfWN9ew+vo8d5Tn/KuDOAuDn5ktScGLTEMrm1oD77NuD7Cs+QZ8YIyP7hN6M6CIi16zZYq6yspLY\n2Nimx35+flRUVDQVc2lpaSQmJhISEtL0GkdHRzw8PABYvHgxycnJODpemBnw73//OwsWLKBDhw48\n88wz+Pn5tegBibQ0L3dn/uvWbgyKD+JvK/P5NvsIO/ZXcPewTgyMC9RisCIiNuJiAZdVnk1u5Z5L\nC7iQfheGUHo3X8BdlF1QyZdbCjG1d+e/xnbT972I2JzrngDFbDY3/X3ixAnS0tJYsGABZWVll712\n9erVLF68mPfffx+A22+/nfbt29OtWzfmz5/PG2+8wf/8z//84Gf5+nrg5NQy08Mbjd4tsp+bQVkt\n40azGo3eJMaHsOzbA/xjVT7vLd9Dxt5yZt7Zk7CAlm+HttS2N5OyWo695ZXW4VxjPXuO55NVnnNJ\nAdfBzZdBIf1JMMUT7h163YVY5cla/vLFbpwcHZh5RxwebpozTkRsT7PfTCaTicrKyqbH5eXlGI1G\n4MK9ccePH+f+++/n3LlzFBUVMXfuXFJTU9mwYQNvv/02f/nLX/D2vtDBDxgwoGk/w4cP59lnn73q\nZ1dVnfkxx3QZo9GbiopTLbIvS1NWy2jJrIPjAukW1o5/rt7Pjv2VPPHyWsb0C2f8wEhcnVvu5ENb\nbFtLU1bLaYm8KgblWp1rrGf38fymIZRnG88B0MHNj8EhA+ht6vGjCriLGhrP8/bSPE7XNfDAmBjC\nLXDCTkSkJTRbzCUlJfH6668zefJk8vLyMJlMTUMsx4wZw5gxYwA4fPgwv/nNb0hNTeXUqVO8+OKL\n/PWvf22auRLgiSee4KmnniIsLIyMjAw6d+5socMSsSz/du48cWc8O/ZVsHD1PpanF5Kxu4wpo7oQ\nH6216UREWtq5xnoyDu9g3f4Mco/tuaSASw6JJ8EUT5h3SIsMhfxk7QG+O1LNgNgAknsG3/D+REQs\npdliLiEhgdjYWCZPnozBYGDOnDmkpaXh7e1NSkrKFd/z5ZdfUlVVxS9+8YumbS+88AL3338/v/jF\nL3B3d8fDw4Pnn3++5Y5ExAp6dzHSLdKXZZsOsSqzmD99ksMtMUbuHdkFX29Xa8cTERt0+vRpZs+e\nzcmTJ6mvr2fWrFmcO3eO+fPn4+zsjJ+fHy+99BKurvoOOdd4jrxj/7oCd2wP5/5VwPlboIC7aNve\ncr7eVkxQBw+mjo7RfXIiYtOuaQD4r371q0sed+3a9bLXhIaGNq0xd88993DPPfdc9prg4GA+/fTT\nH5NTxGa5uThdmAwlNpC/rcpnW34Fuw4e547BHRnRJwRHB61NJyL/9tlnnxEVFcWTTz5JWVkZDzzw\nAAEBAU23JfzmN79h1apV3HbbbdaOahUXC7is8mxyj+39dwHn3oFBkbcQ4xVDmFfLFnAXlVWdYcGK\nPbg4OzDzjh64ueg+ORGxbfqWEmkhoSYvnr4/gY05pXyytoCP1uxn865Spo6J0bpEItLE19eX/Px8\nAKqrq/H19eWDDz4AoKGhgYqKCgICAqwZ8aY713iO3GN72fGvSUzOna8HwOjegd6mC1fgQr2CMZl8\nLHYvaX1DI299lkvt2UZ+Or4bIf6eFvkcEZGWpGJOpAU5GAwk9wxuWptu066jzP3bdob2DuHOIR3x\ncNPadCJt3bhx40hLSyMlJYXq6mreeecd4MJSP/PmzWP48OEkJiZaOaXlnW08R96xvWSV55B3xQKu\nJ6FeQTdtmOM/V++nqLyG5J5BDIwLuimfKSJyowzm7681YGNa6uybPc0Kp6yWYa2s+UVVfLhqH0cq\nT+Pj4cw9IzrTv3tAsz9O1LaWoayWo9ksr93SpUvZtm0bv//979m7dy+pqamkpaUBF67MzZ49m6FD\nhzY7zLKhobHFlu+5WeoazpJ1JJctxVnsKM1tmsQkyMtE/7AEBoQlENH+x89C+WOt217MK//IIirY\nh5d+ltxiMxOLiFiarsyJWFBMuC/P/ldfVmYW8fmmQ7z7+W425pQyZVQXgjpoCI9IW5SVlcWgQYOA\nC/egHz58mHXr1jF06FCcnJwYMWIEmZmZzRZzLbF8z804aXC28Ry5lXsuDKE8tpf6f12BM3n4k2CM\np7cpnpCLV+AaoLKy5qbmPVJ5mjc+ycbNxZFHxnen+oSWRbJlymo59pS3rWW92slOFXMiFubk6MC4\nAZEkdgtg4df7yDlwjDnvZzK2XwTjBkTgojPAIm1KREQE2dnZjB49mpKSEry9vZkzZw4ff/wxAQEB\n5OTkEBUVZe2YN6Su4Sx5x/aQVb6LvO8VcAEexqZ74II9A60+U+TZc428tSSXs/WNzJgYR4Cfh1Xz\niIhcLxVzIjeJsb07P/9JPFn7KvnH6n18vvlQ09p0cR07WDueiNwk99xzD6mpqUyZMoWGhgb+8Ic/\ncO7cOWbNmoWLiwv+/v78/Oc/t3bM61bXcJbcYxeuwF0o4BoACPAwkWDqQW8bKeAuMpvNfLgqn5LK\n04zoE0rfriZrRxIRuW4q5kRuIoPBQJ8YI90jfVm68SCrtx3m1Y+zSexm4p7hnbU2nUgb4OnpyWuv\nvXbZ9iFDhlghzY2pa6gjt3IPWRW72H2FAi7B1JMgz+bvE7aGDTmlbM49SlSQN3cP62TtOCIiP4qK\nORErcHd1YvKIzgyMC+TDlflk7ikn58AxJiV3ZHhCqLXjiYj8oLqGOnb96x643cfzmwq4QA9T0xBK\nWy3gLioqO8XCr/fh6ebEjNvjcHbSeqAiYp9UzIlYUXiAN7+Z2odvs4+weO0B/rF6P5tyj/LopHgC\nfXSVTkRsw/cLuLzj+TRcLOA8A0gw/msIpVeglVNem9qzDby1JJf6hvPMmBiHf3t3a0cSEfnRVMyJ\nWJmDwcDQXiEkdDay6JsC0vOOkvrmJrqEtuO2QVF0j/C16TPcItI61TbUsatyNzvKd7H7ewVckGfA\nJVfg7InZbGbBir2UVdUytl84vTr5WzuSiMgNUTEnYiN8PF14+LbuDE8IYeW2w2zbU8YrH+0kOtiH\n25Ii6dGxg4o6EbGoiwVcVnkOe47vaxUF3Pd9k1XCtr3ldA5txx3JHa0dR0TkhqmYE7Ex0SHtmNMr\nlK27Svh80yF27K/kT5/kEBHozW0DI+nV2R8HFXUi0kJqG2rJqdjNjooc9hzbR4O5EYBgz0B6m3qQ\nYIon0I4LuIsOllbz0Zr9eLk789jtcTg56j45EbF/KuZEbFRkoA9P3BlPcXkNn28+xPa95byRtotQ\noxe3JUXSJ8aook5EfrSy0+W8v/dDskt3X1LAJZguLOQd6Nl6puo/XVfPm5/lcv68mUcmdNfMwSLS\naqiYE7FxYSYvZk6Mo6TyNMvTL6xN99aSXII6eDB+YCSJ3Uw4OugMs4hcn13H9rD9yC5CvILobYwn\nwdSDgFZUwF1kNpt574s9HKuuY0JSJHFRWtdTRFoPFXMidiLE35NHbovl9qQovkg/RHpuGe9+vpul\nGw8yfkAk/WMDNGxIRK7Z8LDB3BqbzLlTrfsK/8rMYnYWVNItwpcJSVHWjiMi0qL0y0/EzgT4eTB9\nXHeef7Q/Q3oFc+xkHe9/uYfU+VtYt7OE+obz1o4oInbAweBAOzcfa8ewqP2HT7B43QHaebnwyIRY\nHBxad+EqIm2PijkRO2Vs784DY7rywmMDGJ4Qwomac/ztq3yefiedNdsPU9/QaO2IIiJWU33mHG8v\nzcOMmccmxNLO08XakUREWpyKORE75+fjxpRRMbw4YwCj+oZxuraehV/v46m301mVWcTZehV1ItK2\nnD9v5t3Pd1N16iyTkjsSE+5r7UgiIhahYk6klWjv5crkEZ15ccZAxvYPp+5cIx99U8BTb23myy2F\n1J5tsHZEEZGb4ov0Q+QdPE58dAfG9o+wdhwREYtRMSfSyvh4unDX0E68NGMg4wdG0tB4nsXrDvDU\nW5tZtukgZ+rqrR1RRMRidh86ztINB+ng48pPx3fXEi4i0qppNkuRVsrL3ZlJyR0ZkxjG6u2H+Xpr\nMUs2HGRlZjEj+4SS0jcML3dna8cUEWkxJ2rOMn9ZHg4OBh6bGKfvOBFp9VTMibRyHm7OTEiKIuWW\nMNbuKOGrjCI+33yIVduKGZ4Qwui+4fhoYgARsXON58/zztI8qs/Uc++IzkQHt7N2JBERi1MxJ9JG\nuLs6cWv/CEYkhLJu54WibsWWItZsO8zQ3iGM6RdOey9Xa8cUEflRlmw4SH7xCfp0MTLyllBrxxER\nuSlUzIm0Ma4ujoxODGdY7xA25JTy5ZZCVm0t5pusEob0DGZs/3D8fNysHVNE5JrlHKhkeXohxvZu\n/Net3TDoPjkRaSNUzIm0US7OjozoE0pyz2A27bpQ1K3JOsy6nSUMig9iXP8I/Nu7WzumiMhVHTtZ\nx7uf78bJ0YGZE3vg4aafNiLSdugbT6SNc3ZyYGjvEAbFB5Ged5Tl6YWs33mEjTmlDIgNZNzACAJ8\nPawdU0TkMg2N53l7aS6n6xqYNjqGiEBva0cSEbmpVMyJCABOjg4Mjg9mYFwgmXvK+WLzITbuKmVT\nbin9ugcwfkAkwf6e1o4pItJk8boDHDhSTf/YAIb0CrZ2HBGRm+6airm5c+eSnZ2NwWAgNTWV+Pj4\ny17zyiuvsHPnTj788MMffE9paSlPPfUUjY2NGI1GXnrpJVxcNIueiC1xdHBgQGwg/boFsC3/QlG3\nJa+MjLwy+nQ1cdvASMJMXtaOKSJt3Pb8ClZtLSaogwfTRsfoPjkRaZOaXTQ8MzOTwsJCFi1axHPP\nPcdzzz132WsKCgrYunVrs++ZN28e9913H//4xz+IiIhg8eLFLXgoItKSHBwMJHYL4NmHEnl8Ug/C\nA7zZtrecOe9n8vqnORQePWXtiCLSRpVXneH9L/fg4uTAzIlxuLlooJGItE3NFnPp6emMHDkSgOjo\naE6ePElNTc0lr/njH//IL3/5y2bfk5GRwYgRIwAYNmwY6enpLXYgImIZDgYDCV2M/M+Dt/CLu+Lp\nGOzDjv2V/O9ft/KnT7LZW3jc2hFFpA2pb2jkzSW51J5tYOroGEKMGikgIm1Xs6eyKisriY2NbXrs\n5+dHRUUFXl4XvjzT0tJITEwkJCSk2ffU1tY2Davs0KEDFRUVLXYgImJZBoOB+Gh/enTswO5DVXy+\n6SA5B47x63kbiI305bakKLqEtbd2TBFp5f65poCishoGxweR1CPI2nFERKzqusclmM3mpr9PnDhB\nWloaCxYsoKys7Jrec7Vt/8nX1wMnJ8frjXhFRqP9zHClrJZhT1nBtvOaTD4MTYxg14FKPlqVT05B\nJXmHqoiL7sDklBjiO/nb7P0rttyu/8mesoL95RX7sy7rMOt2lBBq9OL+lC7WjiMiYnXNFnMmk4nK\nysqmx+Xl5RiNRgC2bNnC8ePHuf/++zl37hxFRUXMnTv3B9/j4eFBXV0dbm5ulJWVYTKZrvrZVVVn\nfuxxXcJo9Kaiwj7u71FWy7CnrGA/eQN9XHluRhLpOw6zbPNBcg8c43cHNtMppB23JUUSF+VnU0Wd\nvbQr2FdWaJm8KgblakqPnebPn+zEzcWRmXfE4eLcMid7RUTsWbP3zCUlJbFy5UoA8vLyMJlMTUMs\nx4wZw5dffsnHH3/MG2+8QWxsLKmpqT/4noEDBzZtX7VqFYMHD7bUcYnITdQptB3/fXcvnnngFnp1\n8qeg5CT/7+Nsfv/BNnbsr7imK/EiIj/kbH0jb36WS925Rh4c25VAP619KSIC13BlLiEhgdjYWCZP\nnozBYGDOnDmkpaXh7e1NSkrKNb8H4IknnmD27NksWrSI4OBgJk6c2LJHIyJWFRXkw89+Ek9R2Sk+\n33yI7fkVvP7pLsJMXtw2MJKEGCMONnSlTkTsw99X5VNSeZrxSVEkdguwdhwREZtxTffM/epXv7rk\ncdeuXS97TWhoaNMac1d6D1wYsrlgwYLrzSgidiY8wJtZd/SgpKKGL9ILydxTxptLcgnx92TcwAgS\nuwbg4KCiTkSatyH7CJt2HSUy0JuHJsRyooVuwRARaQ2aHWYpIvJjhRi9eHRCLM893J+kuEBKj51h\n/rLd/PYvGWzaVUrj+fPWjigiNqy4vIa/f70PD1cnZkyMw7mFJkUTEWktVMyJiMUF+nkwfXx35j7a\nn+SeQVSeqOW95Xv4zTtb+Db7CA2NKupE5FK1Zxt487Nd1DecZ/r4bhjbu1s7koiIzbnupQlERH4s\nU3t3HhzbjdsGRvFlRiEbso/w1xV7+XzTQcb2j2BwfJDOvEurd/r0aWbPns3Jkyepr69n1qxZGI1G\n/u///g8HBwd8fHx45ZVXcHdvu8WL2Wzmg6/2UlZVy5h+4fTubLR2JBERm6RiTkRuug7t3Jg6Kobx\nAyJZkVHI+p1H+PuqfXyx+RBj+0WQ3CsYV007Lq3UZ599RlRUFE8++SRlZWU88MAD+Pv78/TTTxMf\nH88LL7xAWloa999/v7WjWs3aHSVk7imnU2g7JiV3tHYcERGbpWJORKzG19uV+0Z2YdyASFZmFrE2\nq4R/rtnP8vRDjO4XzrDeIbi56GtKWhdfX1/y8/MBqK6uxtfXl7fffrtp2R8/Pz9OnDhhzYhWdbC0\nmo/W7MfL3ZnHJsTi5Kg7QkREfoi+IUXE6tp5unD3sE68OGMA4wdGUN94nk/WHuCpt9L5fPMhztQ1\nWDuiSIsZN24cR44cISUlhSlTpjB79uymQu7MmTMsXbqUMWPGWDmldZyuq+etJbk0Npp5ZEJ3/Hzc\nrB1JRMSm6ZS3iNgMbw8XJiVHMzoxnNXbDvP11mI++/Y7VmYUMfKWUFL6huHp5mztmCI3ZOnSpQQH\nB/Pee++xd+9eUlNTSUtL48yZM8yYMYOHHnqI6OjoZvfj6+uBUwvcY2o0et/wPlqC2WzmnQWZVJ6s\n456ULgxLjLzi62wl77VQVstQVsuxp7zKeoGKORGxOZ5uztw+KIpRfcP4JuswKzOLWbbpEKu2FjOi\nTyij+obh7eFi7ZgiP0pWVhaDBg0CLqzbWl5ezrlz55g5cybjx49n0qRJ17SfqhZYb81o9Kai4tQN\n76clfJVRREbeUbpF+JLSO+SKuWwpb3OU1TKU1XLsKW9by3q1YlDDLEXEZrm7OjFuQCQvzRjI3cM6\n4eLsyPL0Qn791mYWfbOfkzVnrR1R5LpFRESQnZ0NQElJCZ6enrz33nskJiZy1113WTmddew/fILF\n6w7QztOFRybE4uBgsHYkERG7oCtzImLzXF0cGdMvnOEJIazPPsJXGUWszCzmm6wShvQMZmz/CHy9\nXa0dU+Sa3HPPPaSmpjJlyhQaGhp49tln+fWvf01oaCjp6ekA9OvXj8cff9zKSW+O6jPneHtpHmbM\nPHZ7LO08ddVdRORaqZgTEbvh4uxIyi1hDO0VwsZdpXyZfojV2w+zbmcJg+KDubV/OP7t2u7aXGIf\nPD09ee211y7ZtnHjRiulsa7zZjN/+Xw3VafOcueQjsSE+1o7koiIXVExJyJ2x9nJgWG9QxgcH8Tm\n3KMsTz/Euh0lbMg+woC4QMYPiMDk62HtmCLSjOXpheQePE6Pjh0Y2z/C2nFEROyOijkRsVtOjg4k\n9wwmqUcgGbvL+GJzIRtzStm86yj9ugcwfmAEQR08rR1TRK5gT2EVSzZ8h5+PKw/f1h0Hg+6TExG5\nXirmRMTuOTo4MDAuiP7dA9mWX87nmw+RnneULXlH6dvNxPiBkYQavawdU0T+5UTNWd5ZloeDwcCM\n2+PwcteSIyIiP4aKORFpNRwcDCR2C+CWriZ27Kvg802HyNxTTuaecvp0MTJ1XHd8XG98XS4R+fEa\nz5/nnaV5VJ8+x+QRnYkOaWftSCIidkvFnIi0Og4GA31iTCR0MZJ94BifbzrE9n0VbN+3nthIX8b0\nj6B7hC8GDesSuemWbjxIfvEJEroYSbkl1NpxRETsmoo5EWm1DAYDvTr50zO6A3mHjrN6ewk5BZXk\nHaoiPMCLsf0iuKWrEUcHLbkpcjPkHDjGF5sLMbZ346Fbu+qEiojIDVIxJyKtnsFgIC6qA8MSI8nM\nKWFFRhHb88t5Z1ken653Y3RiOIN6BOHqoiGYIpZyvLqOv3yxGydHAzMn9sDDTffJiYjcKBVzItKm\nRAX5MHNiHGVVZ1iVWczGXaUs/HofSzceZHhCCMP7hOLjoUWLRVpSQ+N53lqaS01tPVNHxxAR6G3t\nSCIirYKKORFpkwJ8PZg6OobbB0XxTdZh1mw/zLJNh/gqo4hB8UGMSgzH1F4LkIu0hMXrDnCgpJr+\n3QMY2ivY2nFERFoNFXMi0qb5eLowcXBHxvaLYEPOEVZmFvNNVglrd5TQt6uJMf3CiQz0sXZMEbu1\nPb+CVVuLCergwbQxMbpPTkSkBamYExEBXF0cGXlLGMMSQti6t5yvthQ1LWvQLcKXsf3CiY3y0w9R\nketQfqKW97/c6omXaQAAGGpJREFUg4uTAzMmxuHmop8dIiItSd+qIiLf4+jgQP/ugfTrFsDuQ1Ws\nyChk96Eq9hRWEWbyYky/cPp2NeHkqBkwRa6mvqGRtz7LpfZsA9PHdSPU6GXtSCIirY6KORGRKzAY\nDMRG+REb5Ufh0VOsyChk695y3v18N2nrDzCqbziDewbpSoPID/jomwIKy04xKD6IpB5B1o4jItIq\n6VeIiEgzIgK9eez2OO4cUsuqrcVsyD7CP9fsZ9mmgwxLCGVkn1B8PDUDpshFGbvLWJtVQqjRk/tT\nulg7johIq6ViTkTkGhnbu3N/ShcmJEWyNquE1dsP88XmQ6zMLCKpRxCjE8MI8PWwdkwRqyo9dpq/\nrtiLq4sjM+/ogauz1m8UEbGUayrm5s6dS3Z2NgaDgdTUVOLj45ue+/jjj1m8eDEODg507dqVOXPm\nsHjxYpYtW9b0mtzcXHbs2MHUqVM5c+YMHh4XfuzMnj2buLi4Fj4kERHL8vZwYcKgKEb3C2fTrlK+\nyihi3Y4S1u8ooU+MkTH9IugYrBkwpe05W9/Im0tyOVvfyGO3xxLop5MbIiKW1Gwxl5mZSWFhIYsW\nLeLAgQOkpqayaNEiAGpra1m+fDkLFy7E2dmZadOmsWPHDu666y7uuuuupvevWLGiaX/PP/88Xbpo\nyIWI2D9XZ0eGJ4QypFcw2/MrWJFRxLb8CrblVxAT1p6x/SPo0VEzYErbsXDVPkoqTjM8IYTEbgHW\njiMi0uo1W8ylp6czcuRIAKKjozl58iQ1NTV4eXnh7u7OBx98AFwo7GpqajAajZe8/89//jMvv/yy\nBaKLiNgGRwcHErsF0Lerib2FVazIKCL34HHyi08QYvRkTGI4/boHaAZMadU25Bxh465SIgK9uWd4\nZ2vHERFpE5ot5iorK4mNjW167OfnR0VFBV5e/55ieP78+fztb39j2rRphIWFNW3PyckhKCjokgJv\n3rx5VFVVER0dTWpqKm5ubi11LCIiVmUwGOgW6Ue3SD+Kyk6xMrOIjN3lvLd8D2nffseovmEk9wzG\n3VW3K0vrcri8hoWr9uHu6sTMiXE4O+nEhYjIzXDdvyjMZvNl2x555BGmTZvGww8/TJ8+fejTpw8A\nixcv5o477mh63bRp04iJiSE8PJw5c+awcOFCpk+f/oOf5evrgZNTy9w4bTR6t8h+bgZltQx7ygr2\nlVdZr/w5feKCKT9+hqUbDrBqSyGLvingi82HuDUpitsGdcTX5+ons+ypXcH+8krLqD3bwJtLcjnX\ncJ4nJsRibO9u7UgiIm1Gs8WcyWSisrKy6XF5eXnTlbYTJ06wf/9++vbti5ubG8nJyWRlZTUVcxkZ\nGfzud79rem9KSkrT38OHD+fLL7+86mdXVZ25vqP5AUajNxUVp1pkX5amrJZhT1nBvvIq69UZgIkD\nIxnZO4S1O0pYva2YT9bs57N1BQyMuzADZlAHT5vIeiNaIq+KQftjNpv54Ku9HD1+hjGJ4fTuYmz+\nTSIi0mKaHQeRlJTEypUrAcjLy8NkMjUNsWxoaODpp5/m9OnTAOzatYuoqCgAysrK8PT0xMXlwtpL\nZrOZBx98kOrqauBCode5s8bUi0jb4OXuzG0DI3lpxkCmjY7Bz8eNb7OP8Lt3M3gjbRcHSk5aO6LI\ndVu3o4TMPeV0CmnHpCEdrR1HRKTNafbKXEJCArGxsUyePBmDwcCcOXNIS0vD29ublJQUZs2axbRp\n03ByciImJoYRI0YAUFFRgZ+fX9N+DAYDd999Nw8++CDu7u4EBATwxBNPWO7IRERskIuzI0N7h5Dc\nM5isfRWsyCgka18FWfsq6BLajjH9Iojv1MHaMUWadehoNf9csx8vd2ceuz1WE/yIiFiBwXylm+Bs\nREsNMbKn4UrKahn2lBXsK6+y3hiz2cy+4hOsyCgi58AxAIL9PblrRBdiw9vZzQ9kDbO8+Vriv+Uf\n++92pq6eZxds5djJOn5xd096dLw5JyBs8f/hH6KslqGslmNPeW0h67p1axg6dESzr5s/fx7jx99J\ncHDIj/6sq/WPmlJNRMSKDAYDMeG+xIT7criihq8yisjYXcZri3bQ3suFlL5hDOkZgoebvq7FNpjN\nZt5bvofKk3WMHxh50wo5ERFbUVp6hNWrV15TMffb3/7WooWnfh2IiNiIUKMXPx3fnUnJHdmYV8aK\n9EN8svYAX2w+xNBeIYy8JQxfb1drx5Q2btXWYnbsr6RreHsmDoqydhwRkZvu1VdfYM+ePAYP7suo\nUWMpLT3Cn/70Js8//39UVJRTW1vLQw89QlLSYKZOncrjj/83a9eu4fTpGoqKCikpOczPfvYkAwYk\n3XAWFXMiIjbGz8eN6RPiGNk7mLU7Svh622FWZBSxamsxA+ICGZMYTrD/5TNgilhaweGTLF53gHae\nLjw6IRYHB4O1I4lIG/fxNwVs3Vveovvs29XE3cM7/eDz9947lbS0j4mKiqao6BBvvvkXqqqOk5jY\nn7Fjx1NScphnnnmapKTBl7yvvLyMl1+ex5Ytm1m69FMVcyIirZmHmzPjBkQyqm8Y6XllrMgoYmNO\nKRtzSunVyZ+x/cPpHNre2jGljTh15hxvLc3lvNnMoxNiaeelq8QiIt26xQLg7e3Dnj15LFuWhsHg\nQHX15bNUx8f3Ai4s/VZTU9Min69iTkTExjk7OZLcM5hB8UHs3F/JioxCdhZUsrOgkk4h7RjbL5ye\nnf1xMOgqiVjGebOZd7/YTdWps0xK7kjXCF9rRxIRAeDu4Z2uehXN0pydnQH4+uuvqK6u5s9//gvV\n1dX89KdTL3uto6Nj098tNQelijkRETvhYDCQ0MVIQhcj+4pP8FVGETsLKnk9bReBfh6M6RfOgNhA\nnJ3sYwZMsR9fpheS+91x4jr6ceuACGvHERGxKgcHBxobGy/ZduLECYKCgnFwcGD9+m+or6+/OVlu\nyqeIiEiL6hLWnp/9JJ7f/7Qfg+KDqDhRy19X7OWptzazPP0QZ+puTicird/ewio+2/Advt6uPDy+\nu64Ai0ibFxERRX7+Xk6f/vdQyaFDh7N58wZ+/vMZuLu7YzKZWLDgXYtn0ZU5ERE7FuLvyUO3duOO\nwR1Zva2YdTtL+HT9dyxPL2RIr2BSbgnDz8fN2jHle06fPs3s2bM5efIk9fX1zJo1i6SkJF599VUW\nL17Mli1brB2xycmas7yzLA8Hg4EZE+Pw9nCxdiQREavz9fUlLW35JduCgoL54IOPmh6PGjUW+Pea\neB07/nsoaMeOnXjjjfktkkXFnIhIK+Dr7cpdwzoxbkAk67NL+HprMSszi1m97TD9uwcwul84oUYv\na8cU4LPPPiMqKoonn3ySsrIyHnjgASZOnEhQUFCL3UPREs6fN/POsjxOnj7H5OGd6BTSztqRRETk\nP6iYExFpRTzcnBjbL4KRfcLYsvsoX2UUsSn3KJtyjxIf3YGx/cLpEtYeg4bKWY2vry/5+fkAVFdX\n4+vry5QpU/Dy8mLevHlWTvdvSzYeZG/RCXp39ielb5i144iIyBWomBMRaYWcnRwYHB9MUo8gcgqO\nsSKjkJwDx8g5cIyOwT6M7RdO785GrRNmBePGjSMtLY2UlBSqq6t555138PK6/qumvr4eODk5Nv/C\nZhiN3pdt2763jC82HyLAz4OnHkjEy935hj+npVwpr61SVstQVsuxp7zKeoGKORGRVszBYKBXZ396\ndfan4PDJC8sa7K/kz5/lEuDrzuh+4STFBeLcAkWBXJulS5cSHBzMe++9x969e0lNTSUtLe2691NV\ndeaGs1y8l+P7jlfX8fLft+PkaODRCd2pramjtqbuhj+rJVwpr61SVstQVsuxp7xtLevVikEVcyIi\nbUSn0HY8ERpP6bHTrMwsZnNuKX/7Kp8l337HiFvCGNY7xKauwLRWWVlZDBo0CICuXbtSXl5OY2Pj\nJesPWUtD43neXppHTW09U0d1ITLQx9qRRETkKrQ0gYhIGxPUwZMHx3blxRkDGTcggvpGM599+x2/\nfnMz/1y9n2MnbeMqTGsVERFBdnY2ACUlJXh6etpEIQeQtv47CkpOktjNxNDeIdaOIyIizVAxJyLS\nRrX3cuXOIdG8PHMg9wzvhIebE19vK+bpd9J59/M8istrmt+JXLd77rmHkpISpkyZwpNPPsmzzz7L\n73//e6ZOnUpNTQ1Tp05lwYIFNz3Xjn0VfJVZRKCfBw+M6apJckREbtBPfnIbp0+ftuhnaJiliEgb\n5+7qxOjEcEb0CSVjdxlfZRSRnldGel4ZcR39GNsvgq7hmgGzpXh6evLaa69dsm3AgAFWSnNB+Yla\n/rJ8Dy5ODsycGIe7q34eiIjYA31bi4gIAE6ODiT1CGJgXCC7vjvGii1F5H53nNzvjhMZ6M3Y/hH0\n6aIZMFub+obzvLUkl9qzDTx0azdCTVqPUETkah566H7mzn2FwMBAjh4t5Te/eRKj0URtbS11dXX8\n8pe/pnv3uJuSRcWciIhcwmAwEB/tT3y0P98dqearjEK251fw1pJcjO3dGJMYTlKPIFycbeM+L7kx\ni77ZT+HRUwzqEcSg+CBrxxERuS5pBV+wo3xXi+6zt6kHkzqN/8Hnk5OHsWnTt9x5591s2LCe5ORh\nREd3Jjl5KNu3b2Xhwg947rmXWjTTD9E9cyIi8oM6Bvsw844ezH2kP0N7h1B16hwfrtrHr9/azLJN\nB6mprbd2RLkBG3aU8E1WCSFGT+4f1cXacURE7MKFYm4DABs3rmfQoCGsX7+GGTOm89Zbr3Py5Mmb\nlkVX5kREpFkBfh5MGx3D7YOiWLP9MGuzDrNkw0G+3FLI4PhgHpkUb+2Icp2OHj/D65/swNXFkZkT\n43DVlVYRsUOTOo2/6lU0S+jYMZpjxyooKzvKqVOn2LBhHf7+Jp555vfs3bubN974003LoitzIiJy\nzdp5ujApuSMvzRzIvSM64+3uzJrth9mw84i1o8l12rqnjNqzjTw4pitBHTytHUdExK4MGDCI+fPf\nZPDgIZw8eYKQkFAA1q9fS0NDw03LoStzIiJy3dxcnEjpG8awhBC+O1JNv54hVB237PTL0rJG3hLG\nkFvC8XHVFTkRkes1ZMgwHnvsIf76139SV1fLH/4wh7VrV3PnnXezevUqli9fdlNyqJgTEZEfzcnR\ngS5h7XFy1EAPe+Pu6kS40ZuKilPWjiIiYne6dYtl/fqMpscLFy5u+nvQoCEAjBs3AU9PT86csdz3\nrHpfERERERERO6RiTkRERERExA6pmBMREREREbFDKuZERERERETs0DVNgDJ37lyys7MxGAykpqYS\nH//v9YQ+/vhjFi9ejIODA127dmXOnDlkZmby85//nM6dOwPQpUsXnnnmGUpLS3nqqadobGzEaDTy\n0ksv4eLiYpkjExERERERacWaLeYyMzMpLCxk0aJFHDhwgNTUVBYtWgRAbW0ty5cvZ+HChTg7OzNt\n2jR27NgBQGJiIvPmzbtkX/PmzeO+++5j7NixvPrqqyxevJj77rvPAoclIiIiIiLSujU7zDI9PZ2R\nI0cCEB0dzcmTJ6mpqQHA3d2dDz74AGdnZ2pra6mpqcFoNP7gvjIyMhgxYgQAw4YNIz09vSWOQURE\nREREpM1ptpirrKzE19e36bGfnx8VFRWXvGb+/PmkpKQwZswYwsLCACgoKOCxxx7j3nvvZdOmTcCF\nK3kXh1V26NDhsv2IiIiIiIjItbnuRcPNZvNl2x555BGmTZvGww8/TJ8+fYiMjOTxxx9n7NixFBcX\nM23aNFatWtXsfv6Tr68HTk6O1xvxioxG7xbZz82grJZhT1nBvvIqq2XYU1awv7wiIiL2rtlizmQy\nUVlZ2fS4vLy8aSjliRMn2L9/P3379sXNzY3k5GSysrLo06cPt956KwDh4eH4+/tTVlaGh4cHdXV1\nuLm5UVZWhslkuupnV1WduZFja2I0elNRYbmV11uSslqGPWUF+8qrrJZhT1mhZfKqGBQREbk+zRZz\nSUlJvP7660yePJm8vDxMJhNeXl4ANDQ08PTTT7Ns2TI8PT3ZtWsXEyZMYNmyZVRUVDB9+nQqKio4\nduwYAQEBDBw4kJUrV3L77bezatUqBg8efNXPbsmO3Z5+JCirZdhTVrCvvMpqGfaUFewvr71rqfa2\nt383e8qrrJahrJZjT3mV9QKD+RrGO7788sts27YNg8HAnDlz2L17N97e3qSkpJCWlsbChQtxcnIi\nJiaG//3f/+X06dP86le/orq6mvr6eh5//HGGDBlCeXk5s2fP5uzZswQHB/P888/j7OxssYMTERER\nERFpra6pmBMRERERERHb0uxsliIiIiIiImJ7VMyJiIiIiIjYIRVzIiIiIiIidkjFnIiIiIiIiB26\n7kXDbdncuXPJzs7GYDCQmppKfHx803ObN2/m1VdfxdHRkeTkZGbNmmXFpFfPOnz4cAIDA3F0vLBg\n+ssvv0xAQIC1ogKwb98+Zs6cyYMPPsiUKVMuec7W2vZqWW2tbV988UW2b99OQ0MDjz76KKNGjWp6\nztba9WpZba1da2trefrppzl27Bhnz55l5syZDBs2rOl5W2rb5rLaWtsC1NXVMX78eGbOnMmkSZOa\ntttSu8rl1Edahj31j6A+0lLspY9U/2hZVukfza1ERkaG+ZFHHjGbzWZzQUGB+e67777k+bFjx5qP\nHDlibmxsNN97773m/fv3WyOm2WxuPuuwYcPMNTU11oh2RadPnzZPmTLF/Lvf/c784YcfXva8LbVt\nc1ltqW3T09PNP/3pT81ms9l8/Phx85AhQy553pbatbmsttSuZrPZvHz5cvP8+fPNZrPZfPjwYfOo\nUaMued6W2ra5rLbWtmaz2fzqq6+aJ02aZP70008v2W5L7SqXUh9pGfbUP5rN6iMtxZ76SPWPlmWN\n/rHVDLNMT09n5MiRAERHR3Py5ElqamoAKC4upl27dgQFBeHg4MCQIUNIT0+3yay2yMXFhXfffReT\nyXTZc7bWtlfLamv69u3La6+9BoCPjw+1tbU0NjYCtteuV8tqi2699VYefvhhAEpLSy85U2drbXu1\nrLbowIEDFBQUMHTo0Eu221q7yqXUR1qGPfWPoD7SGlltjfpHy7FW/9hqhllWVlYSGxvb9NjPz4+K\nigq8vLyoqKjAz8/vkueKi4utERO4etaL5syZQ0lJCX369OHJJ5/EYDBYIyoATk5OODld+T8VW2vb\nq2W9yFba1tHREQ8PDwAWL15McnJy01ABW2vXq2W9yFba9fsmT57M0aNHefvtt5u22VrbXnSlrBfZ\nUtu+8MILPPPMMyxZsuSS7bbarnKB+kjLsKf+EdRHWoo99pHqH1uetfrHVlPM/SezHa2F/p9Zf/az\nnzF48GDatWvHrFmzWLlyJWPGjLFSutbFFtt29erVLF68mPfff9+qOa7FD2W1xXYF+Oijj9izZw+/\n/vWvWbZsmdU7z6v5oay21LZLliyhV69ehIWFWeXzpeWoj5QrscW2VR9pGeofW5Y1+8dWM8zSZDJR\nWVnZ9Li8vByj0XjF58rKyqw6xOBqWQEmTpxIhw4dcHJyIjk5mX379lkj5jWxtbZtjq217YYNG3j7\n7bd599138fb2btpui+36Q1nB9to1NzeX0tJSALp160ZjYyPHjx8HbK9tr5YVbKtt161bx5o1a7j7\n7rv55JNPePPNN9m8eTNge+0ql1IfefPZWrteC1trW/WRLU/9o2VYs39sNcVcUlISK1euBCAvLw+T\nydQ0JCM0NJSamhoOHz5MQ0MDa9euJSkpySaznjp1iunTp3Pu3DkAtm7dSufOna2WtTm21rZXY2tt\ne+rUKV588UXeeecd2rdvf8lzttauV8tqa+0KsG3btqYzo5WVlZw5cwZfX1/A9tr2alltrW3/9Kc/\n8emnn/Lxxx9z1113MXPmTAYOHAjYXrvKpdRH3ny21q7NsbW2VR9pGeofLcOa/aPBbE9jLZrx8ssv\ns23bNgwGA3PmzGH37t14e3uTkpLC1q1befnllwEYNWoU06dPt9msH3zwAUuWLMHV1ZXu3bvzzDPP\nWPXyd25uLi+88AIlJSU4OTkREBDA8OHDCQ0Ntbm2bS6rLbXtokWLeP3114mKimra1q9fP2JiYmyu\nXZvLakvtChemBv7tb39LaWkpdXV1PP7445w4ccImvw+ay2prbXvR66+/TkhICIBNtqtcTn1ky7On\n/vFa8tpS26qPtAz1j5Z3s/vHVlXMiYiIiIiItBWtZpiliIiIiIhIW6JiTkRERERExA6pmBMRERER\nEbFDKuZERERERETskIo5ERERERERO6RiTkRERERExA6pmBMREREREbFDKuZERERERETs0P8HN+yS\niiNdgdUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f709d6152e8>"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot performance\n",
    "trainer.plot_performance()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 51
    },
    "colab_type": "code",
    "id": "rFYs9dH3ui66",
    "outputId": "0621602f-40c3-4d55-cf6a-a81c503509fe"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test loss: 0.46\n",
      "Test Accuracy: 83.9%\n"
     ]
    }
   ],
   "source": [
    "# Test performance\n",
    "trainer.run_test_loop()\n",
    "print(\"Test loss: {0:.2f}\".format(trainer.train_state['test_loss']))\n",
    "print(\"Test Accuracy: {0:.1f}%\".format(trainer.train_state['test_acc']))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "vKDmGXAyukpV"
   },
   "outputs": [],
   "source": [
    "# Save all results\n",
    "trainer.save_train_state()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "P4yiDxOavmJy"
   },
   "source": [
    "So you can see that using GloVe embeddings but not freezing them resulted in the best results on the test dataset. Different tasks will yield different results so you need to choose whether or not to freeze your embeddings based on empirical results."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "F0yKVPF8xBQK"
   },
   "source": [
    "# TODO"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "A0e8p2LkxD6x"
   },
   "source": [
    "* when to use skip-gram/CBOW\n",
    "    * According to Mikolov:\n",
    "        \n",
    "        Skip-gram: works well with small amount of the training data, represents well even rare words or phrases.\n",
    "        \n",
    "        CBOW: several times faster to train than the skip-gram, slightly better accuracy for the frequent words\n",
    "* Loading word2vec\n",
    "* interpretable conv filters since we apply it on words here\n",
    "* [contextualized word embeddings](https://arxiv.org/abs/1607.00578)"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "name": "12_Embeddings",
   "provenance": [],
   "toc_visible": true,
   "version": "0.3.2"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
